From a9db8a4c71d71cf9487ec3d19a254175b09329dd Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Fri, 30 Oct 2026 21:57:09 -0400 Subject: [PATCH 01/27] chore(docs): centralise changelog entries for unreleased versions Pull requests for user-visible changes each adding a bullet to the unreleased version's changelog file conflict on every merge. CONTRIBUTING.md now says the maintainer writes those entries centrally, and that such PRs leave the file out of the diff. The PR template's changelog box and the generator that annotates it follow the same rule. Refs #2449 --- .github/PULL_REQUEST_TEMPLATE.md | 2 +- CONTRIBUTING.md | 12 +++++++++--- tests/project/test_python_versions_update_1660.py | 4 +++- util/update_python_versions.py | 2 +- 4 files changed, 14 insertions(+), 6 deletions(-) diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index f6ff3f0bb1..ed810c099a 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -7,7 +7,7 @@ - [ ] [Searched](https://github.com/JarryShaw/PyPCAPKit/pulls?q=is%3Apr) for similar pull requests - [ ] Followed the [coding style](https://github.com/JarryShaw/PyPCAPKit/blob/main/CONTRIBUTING.md#coding-style) (`make pylint`, `make mypy`, `make isort`) - [ ] `make test` passes, and a test case covers the change -- [ ] Added a [changelog entry](https://github.com/JarryShaw/PyPCAPKit/blob/main/CONTRIBUTING.md#changelog-entries) under `docs/source/changelog/` and regenerated `CHANGELOG.md`, if the change is user-visible +- [ ] Described the user-visible change below and left its [changelog entry](https://github.com/JarryShaw/PyPCAPKit/blob/main/CONTRIBUTING.md#changelog-entries) to the maintainer, keeping `docs/source/changelog/` and `CHANGELOG.md` out of the diff, if the change is user-visible ### What is the purpose of your *pull request*? diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 24d47a992b..91c421a11f 100755 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -25,7 +25,8 @@ A rough outline: - Create a topic branch, usually from `main`. - Make commits of logical units, and add a test case if the change fixes a bug or adds functionality. - Run the tests and make sure they pass (see below). -- Add a changelog entry if the change is user-visible (see below). +- If the change is user-visible, describe it in the pull request and leave its changelog entry to + the maintainer (see below). - Make sure your commit messages are in the proper format (see below). - Push your changes to a topic branch in your fork of the repository. - Submit a pull request to the repo. @@ -58,10 +59,15 @@ shapes of it at collection time and say what to do; `tests/_tiers.py` has the re Entries live in `docs/source/changelog/.rst`, one file per version, listed newest-first in the toctree of `docs/source/changelog.rst`. That tree is the single source of the project's history. +**Pull requests for user-visible changes do not add their own bullet.** While a version is +unreleased, its entry is written centrally by the maintainer once a change is ready to merge or has +merged. Pull requests that each edit the same file conflict on every merge, so leave +`docs/source/changelog/.rst` and `CHANGELOG.md` out of the diff. + **`CHANGELOG.md` is generated — never edit it by hand.** `util/changelog_md.py` derives it from the newest entry alone, because its consumers (the `Create Release` workflow's release body and the -source distribution) read Markdown rather than reStructuredText. Add your bullet to the `.rst` -entry, then regenerate: +source distribution) read Markdown rather than reStructuredText. The maintainer adds the bullet to +the `.rst` entry, then regenerates: ```shell python util/changelog_md.py # rewrite CHANGELOG.md from the newest entry diff --git a/tests/project/test_python_versions_update_1660.py b/tests/project/test_python_versions_update_1660.py index 7e42138870..9a86e3911a 100644 --- a/tests/project/test_python_versions_update_1660.py +++ b/tests/project/test_python_versions_update_1660.py @@ -876,7 +876,9 @@ def test_the_body_lists_what_the_owner_does_by_hand(self) -> None: body = module.fill_template((ROOT / module.TEMPLATE).read_text(encoding='utf-8'), description, 'failure') self.assertIn('- [x] `ci` — workflows or build tooling', body) self.assertEqual(body.count('- [x]'), 1) - self.assertIn('regenerated `CHANGELOG.md`, if the change is user-visible — N/A — centralised in #657', body) + self.assertIn('- [ ] Described the user-visible change below and left its [changelog entry]', body) + self.assertIn('keeping `docs/source/changelog/` and `CHANGELOG.md` out of the diff, ' + 'if the change is user-visible — N/A — centralised in #657', body) self.assertIn('### Description of your *pull request* and other information', body) self.assertNotIn('  Explanation of your', body) self.assertIn(f'| `3.11` | supported | KTLO | {plan["changes"][0]["reason"]} |', body) diff --git a/util/update_python_versions.py b/util/update_python_versions.py index f654da8321..52a65635fe 100644 --- a/util/update_python_versions.py +++ b/util/update_python_versions.py @@ -1298,7 +1298,7 @@ def fill_template(template: 'str', description: 'str', audit_outcome: 'str') -> text = re.sub(r'(?m)^- \[ \] `ci`', '- [x] `ci`', template) audit = {'success': 'passed', 'failure': 'failed'}.get(audit_outcome, audit_outcome) text = re.sub(r'(?m)^(- \[ \] `make test` passes.*)$', rf'\1 -- not run; the audit {audit} (below)', text) - text = re.sub(r'(?m)^(- \[ \] Added a \[changelog entry\].*)$', r'\1 — N/A — centralised in #657', text) + text = re.sub(r'(?m)^(- \[ \] Described the user-visible change.*)$', r'\1 — N/A — centralised in #657', text) placeholder = re.search(r'(?m)^  Explanation of your \*pull request\*.*$', text) if placeholder is None: return text.rstrip('\n') + '\n\n' + description + '\n' From db01d1b2f6e9213d5df1af941725b956facf83e5 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 01:34:39 -0500 Subject: [PATCH 02/27] docs(changelog): consolidate the 1.6.0 entries (#2449) - add entries for the user-visible changes merged since 1.5.1 that had none: LinuxSLL, ICMP/ICMPv6, IGMP, TLS/DTLS, SSLv2, DTLS reassembly, IP_Pseudo, the IPv6 extension-header walk and the IPv6_Ext id() and membership fixes - add the Radiotap entry from #2509 - drop the FRAD, Netlink and GRE entries' stale "remain Raw" and "tracked separately" notes, and cite each entry's own PR and issue - record the centralised-changelog rule this PR introduces - regenerate CHANGELOG.md; util/changelog_md.py --check passes --- CHANGELOG.md | 32 ++++- docs/source/changelog/1.6.0.rst | 206 +++++++++++++++++++++++++------- 2 files changed, 191 insertions(+), 47 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d88b4c1dad..81155f3efb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,12 +1,36 @@ ## 1.6.0 -- unreleased -Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. +Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, and stricter handling of a malformed IPv6 extension header. -- **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)). `LinuxSLL`/ `LinuxSLL1`/ `LinuxSLL2` dispatch to it for both the `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` cases -- the same GRE framing either way, the "IP"/"IP6" distinction being only which outer IP version the tunnel carries -- instead of falling back to `Raw`. The Protocol Type field dispatches onward as an EtherType, through the same family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL already share. One of [#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)'s five `arphrd_type` follow-ups to get a dissector (`NETLINK` and `FRAD` are tracked separately on [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507) and [#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)); `IEEE80211_RADIOTAP` remains `Raw`. +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated from a hand-maintained table of `` in `pcapkit.vendor.linux.arphrd.ARPHRD` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). -- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's own kernel/userspace IPC transport (the kernel's own `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)). `LinuxSLL`/ `LinuxSLL1`/ `LinuxSLL2` dispatch to it for the `ARPHRD_NETLINK` case instead of falling back to `Raw`. The byte order the header was written in is ambiguous -- Netlink is kernel/userspace IPC, not wire format -- and is guessed the way Wireshark's own netlink dissector does, by `nlmsg_byteorder()`. Per-family message payloads (routing table entries and the like) and multiple messages packed into one capture are out of scope for this first cut; every message's payload is kept as `Raw`, unconditionally. One of [#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)'s five `arphrd_type` follow-ups to get a dissector (`FRAD` is tracked separately on [#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)); `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` remain `Raw`. +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`. -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)). `LinuxSLL`/ `LinuxSLL1`/ `LinuxSLL2` dispatch to it for the `ARPHRD_FRAD` case instead of falling back to `Raw`; the direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others. The first of [#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)'s five `arphrd_type` follow-ups to get a dissector; the other four (`NETLINK`, `IPGRE`/`IP6GRE`, `IEEE80211_RADIOTAP`) remain `Raw`. +- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: its per-family payload, and any further message in the same capture, are kept as `Raw`. + +- **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. + +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded, and the 802.11 MAC frame behind them, which has no dissector in this package yet, as `Raw`. + +- **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. The checksum is recorded, never verified or computed ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470)). ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. + +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type and length ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's `id()` is its own name alone ([#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. + +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). + +- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`. Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. + +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in: given to the `DTLS` dissector as `reassembly=`, with the 4-tuple as `flow=`, it lets the completing fragment's message decode, for the first record of each datagram only. `Extractor` does not drive it ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496)). + +- **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk, raises `ProtocolError`. It is not a layer on the wire. + +- **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. + +- **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). + +- **Fixed** -- selecting `'IPv6_Ext'` matched none of the dedicated extension headers: `IPv6_Ext.id()` now lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH and ESP do under IPsec ([#2462](https://github.com/JarryShaw/PyPCAPKit/issues/2462), [#2464](https://github.com/JarryShaw/PyPCAPKit/pull/2464)). + +- **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). --- diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 2b5b603f2a..9d75e50f25 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -3,52 +3,172 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had -no dissector at all. +no dissector at all. Alongside it come the Linux cooked-capture headers and a +dissector for each ``arphrd_type`` they do not read as an EtherType, the IP +pseudo-header, and stricter handling of a malformed IPv6 extension header. + +* **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and + :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the + Linux "cooked" capture headers ``tcpdump -i any`` writes, under + ``LINKTYPE_LINUX_SLL`` (113) and ``LINKTYPE_LINUX_SLL2`` (276), on a + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` family base; PCAP and + PCAP-NG frames of either link type dispatch to them (:issue:`2438`, + :pr:`2443`). ``'LinuxSLL'`` selects either version, and each version's own + name selects that version alone. ``protocol_type`` dispatches onward as an + EtherType, except under the ``arphrd_type`` values that carry none -- + ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- + each of which has a dissector of its own, below (:issue:`2460`). The + hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated + from a hand-maintained table of ```` in + :class:`pcapkit.vendor.linux.arphrd.ARPHRD` (:issue:`2495`, :pr:`2497`). + +* **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame + Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`), + which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for + ``ARPHRD_FRAD`` (:pr:`2499`). The direct NLPID and SNAP-encapsulated forms + both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every + Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or + SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + +* **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for + the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC + transport (the kernel's ``netlink(7)`` man page; + :rfc:`3549#section-2.3.2`), which + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for + ``ARPHRD_NETLINK`` (:issue:`2500`, :pr:`2507`). Netlink is IPC rather than a + wire format, so the header's byte order is ambiguous; + :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder` guesses it the way + Wireshark's netlink dissector does. Only the first message's header is + dissected: its per-family payload, and any further message in the same + capture, are kept as :class:`~pcapkit.protocols.misc.raw.Raw`. * **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the Generic Routing Encapsulation base header (:rfc:`2784`) plus the optional Key - and Sequence Number extensions (:rfc:`2890`). - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`/ - :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1`/ - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2` dispatch to it for both - the ``ARPHRD_IPGRE`` and ``ARPHRD_IP6GRE`` cases -- the same GRE framing either - way, the "IP"/"IP6" distinction being only which outer IP version the tunnel - carries -- instead of falling back to - :class:`~pcapkit.protocols.misc.raw.Raw`. The Protocol Type field dispatches - onward as an EtherType, through the same family registry + and Sequence Number extensions (:rfc:`2890`), which + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for both + ``ARPHRD_IPGRE`` and ``ARPHRD_IP6GRE`` -- the same GRE framing either way, + the two differing only in the tunnel's outer IP version (:issue:`2501`, + :pr:`2508`). The Protocol Type field dispatches onward as an EtherType, + through the family registry (:attr:`Link.__proto__ `) - Ethernet, VLAN and LinuxSLL already share. One of :issue:`2460`'s five - ``arphrd_type`` follow-ups to get a dissector (``NETLINK`` and ``FRAD`` are - tracked separately on :pr:`2507` and :pr:`2499`); ``IEEE80211_RADIOTAP`` - remains :class:`~pcapkit.protocols.misc.raw.Raw`. + Ethernet, VLAN and LinuxSLL share. A non-zero Version, :rfc:`2637`'s + Enhanced GRE, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` rather than + misread. -* **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for - the fixed 16-octet ``nlmsghdr`` framing of Linux's own kernel/userspace IPC - transport (the kernel's own ``netlink(7)`` man page; :rfc:`3549#section-2.3.2`). - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`/ - :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1`/ - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2` dispatch to it for the - ``ARPHRD_NETLINK`` case instead of falling back to - :class:`~pcapkit.protocols.misc.raw.Raw`. The byte order the header was written - in is ambiguous -- Netlink is kernel/userspace IPC, not wire format -- and is - guessed the way Wireshark's own netlink dissector does, by - :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder`. Per-family message - payloads (routing table entries and the like) and multiple messages packed into - one capture are out of scope for this first cut; every message's payload is kept - as :class:`~pcapkit.protocols.misc.raw.Raw`, unconditionally. One of - :issue:`2460`'s five ``arphrd_type`` follow-ups to get a dissector (``FRAD`` is - tracked separately on :pr:`2499`); ``IPGRE``/``IP6GRE`` and - ``IEEE80211_RADIOTAP`` remain :class:`~pcapkit.protocols.misc.raw.Raw`. +* **Added** -- :class:`~pcapkit.protocols.link.radiotap.Radiotap`, a dissector + for the radiotap header () capture tools prepend to + an 802.11 frame, which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` + dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`). + It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) + and records every present-flags word, following bit 31 from one word to the + next. The optional fields those words select are kept whole rather than + decoded, and the 802.11 MAC frame behind them, which has no dissector in this + package yet, as :class:`~pcapkit.protocols.misc.raw.Raw`. -* **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame - Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`). - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`/ - :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1`/ - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2` dispatch to it for the - ``ARPHRD_FRAD`` case instead of falling back to - :class:`~pcapkit.protocols.misc.raw.Raw`; the direct NLPID and SNAP-encapsulated - forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every - Ethernet/VLAN/LinuxSLL frame shares) ARP among others. The first of :issue:`2460`'s - five ``arphrd_type`` follow-ups to get a dissector; the other four - (``NETLINK``, ``IPGRE``/``IP6GRE``, ``IEEE80211_RADIOTAP``) remain - :class:`~pcapkit.protocols.misc.raw.Raw`. +* **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` (:rfc:`792`) + and :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` (:rfc:`4443`), on an + abstract :class:`~pcapkit.protocols.internet.icmp.ICMP` base, for IP + protocols 1 and 58, which were kept as + :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2436`, :pr:`2446`); a + protochain names ``ICMPv6`` where it named ``IPv6_ICMP``. Echo, Destination + Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and + ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD + included, keeps its body raw. The checksum is recorded, never verified or + computed (:issue:`2470`). ICMPv4 takes the plain ``ICMP`` of :rfc:`792` as + its ``id()`` and alias. + :func:`~pcapkit.foundation.registry.protocols.register_icmp_type` registers a + message-body parser. The type and code registries are + :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. + +* **Added** -- IGMPv1 (:rfc:`1112`), IGMPv2 (:rfc:`2236`) and IGMPv3 + (:rfc:`3376`), as :class:`IGMPv1 `, + :class:`IGMPv2 ` and + :class:`IGMPv3 ` on an abstract + :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, + which picks the version from the message type and length (:issue:`2439`, + :issue:`2448`, :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source + list and a Membership Report its group records, and an IGMPv3 Query's Max + Resp Code and QQIC are decoded from their floating-point form. Any other + message, a 9- to 11-octet Query included, is kept as + :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of + :rfc:`1112` as its ``id()`` and alias, and IGMPv2's ``id()`` is its own name + alone (:pr:`2506`). The registries are :mod:`pcapkit.const.igmp`: the types + crawled from IANA, the group record types from a hand-maintained table. + +* **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS + 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` + (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract + :class:`~pcapkit.protocols.application.ssl.SSL` base (:issue:`2437`, + :pr:`2441`). Each reads every version of its record layer and reports it + through ``alias``, as :class:`~pcapkit.protocols.link.arp.ARP` does by + operation, telling 1.3 apart by its ``supported_versions`` extension; + ``id()`` stays ``('TLS',)`` or ``('DTLS',)``. A cleartext handshake record is + decoded -- ClientHello and ServerHello down to their cipher suites, + compression methods and extension types, any other message at type and + length -- as are an alert and ChangeCipherSpec, while encrypted records stay + opaque octets. :class:`~pcapkit.protocols.application.tls.SSLv3` exists for + explicit construction only. A record split across TCP segments is not + reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also + reads and builds DTLS 1.3's unified-header records + (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays + Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, + :pr:`2494`). The content and handshake types are :mod:`pcapkit.const.tls`, + crawled from IANA (:issue:`2495`, :pr:`2503`). + +* **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector + for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares + nothing with TLS's (:issue:`2457`, :pr:`2466`). It is bound to no port: on + TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete + CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else + stays :class:`~pcapkit.protocols.misc.raw.Raw`. Its constants are + :mod:`pcapkit.const.sslv2`, from a hand-maintained table. + +* **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as + ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented + across records (:rfc:`6347#section-4.2.3`), keyed by the UDP 4-tuple and the + epoch, accepting overlapping fragments and holding a buffer until it + completes (:issue:`2440`, :pr:`2451`). It is opt-in: given to the + :class:`~pcapkit.protocols.application.dtls.DTLS` dissector as + ``reassembly=``, with the 4-tuple as ``flow=``, it lets the completing + fragment's message decode, for the first record of each datagram only. + :class:`~pcapkit.foundation.extraction.Extractor` does not drive it + (:issue:`2496`). + +* **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, + the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an + upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit + values, or both (:issue:`2471`, :pr:`2479`). From a layer it takes the final + destination of a source route or of a Routing header with segments left, a + Home Address option's address as the IPv6 source, and the upper-layer + length, from UDP's Length or a jumbogram's Jumbo Payload Length where those + apply; a fragment of a larger packet, or a header it cannot walk, raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError`. It is not a layer on the + wire. + +* **Changed** -- **a breaking change to** the IPv6 extension-header walk: a + known extension header whose dedicated parser raises is now the upper layer, + parsed once as :class:`~pcapkit.protocols.misc.raw.Raw`, and the walk stops + there, as :rfc:`8200#section-4` has a node discard such a packet + (:issue:`2477`, :pr:`2480`). It used to fall back to the generic + :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and walk on; + ``IPv6_Ext`` now parses only Shim6, which has no dedicated parser, and + ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. + +* **Changed** -- a pull request no longer adds its own changelog bullet: an + unreleased version's entries are written centrally by the maintainer, and + ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, + :pr:`2483`). + +* **Fixed** -- selecting ``'IPv6_Ext'`` matched none of the dedicated + extension headers: + :meth:`IPv6_Ext.id() ` now + lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, + AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH + and ESP do under IPsec (:issue:`2462`, :pr:`2464`). + +* **Fixed** -- a membership test such as ``'ESP' in layer`` on an extension + header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, + IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised :exc:`AttributeError`. + The layer now holds :class:`~pcapkit.protocols.misc.null.NoPayload`, the + upper layer belonging to the enclosing IPv6 header (:issue:`2484`, + :issue:`2487`, :pr:`2486`, :pr:`2488`). From 4b93dc819dcdc837f416f73d059197eca67b1370 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 02:22:17 -0500 Subject: [PATCH 03/27] docs(changelog): fold the IGMP id() fix and review wording into 1.6.0 (#2449) - IGMP: add IGMPv3.id() ('IGMPv3',) and the base's ('IGMP', 'IGMPv2', 'IGMPv3') from #2518; version selection is by message type, except a Membership Query, split by length and, at 8 octets, by Max Resp Time. - LinuxSLL: ARPHRD names only the values its crawler's table lists; any other, ARPHRD_ETHER included, reads as Unassigned. - IPv6 extension-header walk: the library no longer passes IPv6_Ext.read() an error=, so IPv6_Ext.error is None unless a caller supplies one; the Raw layer's info.error holds the failure. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 6 +++--- docs/source/changelog/1.6.0.rst | 38 ++++++++++++++++++++++----------- 2 files changed, 28 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 81155f3efb..58cccd4ef9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, and stricter handling of a malformed IPv6 extension header. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated from a hand-maintained table of `` in `pcapkit.vendor.linux.arphrd.ARPHRD` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). - **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`. @@ -14,7 +14,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. The checksum is recorded, never verified or computed ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470)). ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type and length ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's `id()` is its own name alone ([#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. - **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). @@ -24,7 +24,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk, raises `ProtocolError`. It is not a layer on the wire. -- **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. +- **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 9d75e50f25..1260398f0b 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -18,9 +18,11 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. EtherType, except under the ``arphrd_type`` values that carry none -- ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- each of which has a dissector of its own, below (:issue:`2460`). The - hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated - from a hand-maintained table of ```` in - :class:`pcapkit.vendor.linux.arphrd.ARPHRD` (:issue:`2495`, :pr:`2497`). + hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by + :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It + names only the ```` values that table lists, so any other, + ``ARPHRD_ETHER`` included, reads as ``Unassigned`` (:issue:`2495`, + :pr:`2497`). * **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`), @@ -85,15 +87,19 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. :class:`IGMPv2 ` and :class:`IGMPv3 ` on an abstract :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, - which picks the version from the message type and length (:issue:`2439`, - :issue:`2448`, :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source - list and a Membership Report its group records, and an IGMPv3 Query's Max - Resp Code and QQIC are decoded from their floating-point form. Any other - message, a 9- to 11-octet Query included, is kept as - :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of - :rfc:`1112` as its ``id()`` and alias, and IGMPv2's ``id()`` is its own name - alone (:pr:`2506`). The registries are :mod:`pcapkit.const.igmp`: the types - crawled from IANA, the group record types from a hand-maintained table. + which picks the version from the message type alone except for a Membership + Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 + otherwise, one of 12 or more IGMPv3 (:issue:`2439`, :issue:`2448`, + :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source list and a + Membership Report its group records, and an IGMPv3 Query's Max Resp Code and + QQIC are decoded from their floating-point form. Any other message, a 9- to + 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and + IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and + ``('IGMPv3',)``; the base's is ``('IGMP', 'IGMPv2', 'IGMPv3')``, each + member's canonical name (:issue:`2512`, :pr:`2506`, :pr:`2518`). The + registries are :mod:`pcapkit.const.igmp`: the types crawled from IANA, the + group record types from a hand-maintained table. * **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` @@ -152,7 +158,13 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. (:issue:`2477`, :pr:`2480`). It used to fall back to the generic :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and walk on; ``IPv6_Ext`` now parses only Shim6, which has no dedicated parser, and - ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. + ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. The + library no longer passes + :meth:`IPv6_Ext.read() ` + an ``error=``, so + :attr:`IPv6_Ext.error ` + is :data:`None` unless a caller supplies one; the ``Raw`` layer's + ``info.error`` holds the failure instead. * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and From ccedecf79d6984ca7a7de531aff11b4ed9b4183d Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 12:35:45 -0500 Subject: [PATCH 04/27] docs(changelog): fold main's latest merge wave into 1.6.0 (#2449) - Added: one's-complement checksum verification and computation (#2543), and the IPX and SCTP registration wrappers (#2521). - Folded into unreleased entries: LinuxSLL/FRAD/Netlink registries (#2527), the IGMP class names (#2542), TLS record chaining and the SSLv2 trial's warnings (#2537), DTLS reassembly via Extractor and toolkit (#2528), and IP_Pseudo's walk at any depth (#2543). ICMP drops its stale checksum line. - Breaking: make() checksums, DRARP's module (#2540), IPv6 overlapping fragments (#2569), TCP_GAP_LIMIT replacing TCP.__window__ (#2579), and Extractor/extract's positional dtls (#2528). - Changed: member order (#2490), conventions (#2542, #2521, #2570), CI -n logical (#2572). Fixed: AppType.get (#2568), trial warnings (#2519, #2577). - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 46 ++++++-- docs/source/changelog/1.6.0.rst | 186 +++++++++++++++++++++++++++----- 2 files changed, 194 insertions(+), 38 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 58cccd4ef9..7240a87efb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,33 +1,59 @@ ## 1.6.0 -- unreleased -Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, and stricter handling of a malformed IPv6 extension header. +Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, and stricter handling of a malformed IPv6 extension header. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`. +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). -- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: its per-family payload, and any further message in the same capture, are kept as `Raw`. +- **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as `Raw` unless `register_netlink()` registers a dissector for its `nlmsg_type` ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. - **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded, and the 802.11 MAC frame behind them, which has no dissector in this package yet, as `Raw`. -- **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. The checksum is recorded, never verified or computed ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470)). ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. +- **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3` on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). -- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`. Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. +- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. -- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in: given to the `DTLS` dissector as `reassembly=`, with the 4-tuple as `flow=`, it lets the completing fragment's message decode, for the first record of each datagram only. `Extractor` does not drive it ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496)). +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). -- **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk, raises `ProtocolError`. It is not a layer on the wire. +- **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. + +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.protocols.misc.checksum.checksum` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. + +- **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. +- **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. + +- **Changed** -- **a breaking change to** `DRARP`: it moves from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp`, with no re-export at the old path, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540)). `pcapkit.DRARP`, `pcapkit.protocols.DRARP` and a parsed RARP's `DRARP` alias are unchanged. + +- **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. + +- **Changed** -- **a breaking change to** TCP reassembly's gap cap: `pcapkit.foundation.reassembly.tcp.TCP.__window__` is removed with no alias, and the cap is `TCP_GAP_LIMIT`, set by `PCAPKIT_TCP_GAP_LIMIT`, 16 MiB by default as before and `-1` for no limit ([#2546](https://github.com/JarryShaw/PyPCAPKit/issues/2546), [#2579](https://github.com/JarryShaw/PyPCAPKit/pull/2579)). It is a safety limit, read each time `TCP.reassembly` runs, and `-1` or a value above the default warns with `ConfigWarning`. + +- **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). + +- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). + +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). + +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). + - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). +- **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). + +- **Fixed** -- under `beholder`, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before being replaced with `Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries ([#2516](https://github.com/JarryShaw/PyPCAPKit/issues/2516), [#2519](https://github.com/JarryShaw/PyPCAPKit/pull/2519)). + +- **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as under `beholder`, and the IPv4 and TCP header check holds its trial's and discards them when it raises ([#2526](https://github.com/JarryShaw/PyPCAPKit/issues/2526), [#2577](https://github.com/JarryShaw/PyPCAPKit/pull/2577)). + - **Fixed** -- selecting `'IPv6_Ext'` matched none of the dedicated extension headers: `IPv6_Ext.id()` now lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH and ESP do under IPsec ([#2462](https://github.com/JarryShaw/PyPCAPKit/issues/2462), [#2464](https://github.com/JarryShaw/PyPCAPKit/pull/2464)). - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 1260398f0b..8fde3ebe0f 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -5,7 +5,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each ``arphrd_type`` they do not read as an EtherType, the IP -pseudo-header, and stricter handling of a malformed IPv6 extension header. +pseudo-header, verified and computed one's-complement checksums, and stricter +handling of a malformed IPv6 extension header. * **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the @@ -17,8 +18,11 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. name selects that version alone. ``protocol_type`` dispatches onward as an EtherType, except under the ``arphrd_type`` values that carry none -- ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- - each of which has a dissector of its own, below (:issue:`2460`). The - hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by + each of which has a dissector of its own, below (:issue:`2460`), reached + through ``LinuxSLL``'s own registry keyed by ``arphrd_type``, which + :func:`~pcapkit.foundation.registry.protocols.register_arphrd` extends + (:issue:`2510`, :pr:`2527`). The hardware types are + :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the ```` values that table lists, so any other, ``ARPHRD_ETHER`` included, reads as ``Unassigned`` (:issue:`2495`, @@ -30,7 +34,9 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. ``ARPHRD_FRAD`` (:pr:`2499`). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or - SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and + :func:`~pcapkit.foundation.registry.protocols.register_frad` registers a + dissector for a further NLPID (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC @@ -41,8 +47,10 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. wire format, so the header's byte order is ambiguous; :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder` guesses it the way Wireshark's netlink dissector does. Only the first message's header is - dissected: its per-family payload, and any further message in the same - capture, are kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + dissected: what follows it, its per-family payload and any further message + in the same capture, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` + unless :func:`~pcapkit.foundation.registry.protocols.register_netlink` + registers a dissector for its ``nlmsg_type`` (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the Generic Routing Encapsulation base header (:rfc:`2784`) plus the optional Key @@ -75,17 +83,17 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. protochain names ``ICMPv6`` where it named ``IPv6_ICMP``. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD - included, keeps its body raw. The checksum is recorded, never verified or - computed (:issue:`2470`). ICMPv4 takes the plain ``ICMP`` of :rfc:`792` as - its ``id()`` and alias. + included, keeps its body raw. ICMPv4 takes the plain ``ICMP`` of :rfc:`792` + as its ``id()`` and alias. :func:`~pcapkit.foundation.registry.protocols.register_icmp_type` registers a message-body parser. The type and code registries are :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. * **Added** -- IGMPv1 (:rfc:`1112`), IGMPv2 (:rfc:`2236`) and IGMPv3 - (:rfc:`3376`), as :class:`IGMPv1 `, - :class:`IGMPv2 ` and - :class:`IGMPv3 ` on an abstract + (:rfc:`3376`), as :class:`~pcapkit.protocols.internet.igmpv1.IGMPv1`, + :class:`~pcapkit.protocols.internet.igmpv2.IGMPv2` and + :class:`~pcapkit.protocols.internet.igmpv3.IGMPv3`, each named after its + module (:issue:`2529`, :pr:`2542`), on an abstract :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 @@ -112,9 +120,13 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay - opaque octets. :class:`~pcapkit.protocols.application.tls.SSLv3` exists for - explicit construction only. A record split across TCP segments is not - reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also + opaque octets. Each whole TLS record in a TCP segment is a layer of its own, + the previous record's payload, so each alias names its own version -- + ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment + (:issue:`2517`, :pr:`2537`). :class:`~pcapkit.protocols.application.tls.SSLv3` + exists for explicit construction only. A record split across TCP segments is + not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also reads and builds DTLS 1.3's unified-header records (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, @@ -124,21 +136,25 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. * **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's (:issue:`2457`, :pr:`2466`). It is bound to no port: on - TCP 443 a record TLS rejects is tried as SSLv2, and kept only as a complete - CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else - stays :class:`~pcapkit.protocols.misc.raw.Raw`. Its constants are - :mod:`pcapkit.const.sslv2`, from a hand-maintained table. + TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and + kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields + agree, so anything else stays :class:`~pcapkit.protocols.misc.raw.Raw`; a + kept one drops the rejected TLS trial's warnings (:issue:`2525`, + :pr:`2537`). Its constants are :mod:`pcapkit.const.sslv2`, from a + hand-maintained table. * **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented across records (:rfc:`6347#section-4.2.3`), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it - completes (:issue:`2440`, :pr:`2451`). It is opt-in: given to the - :class:`~pcapkit.protocols.application.dtls.DTLS` dissector as - ``reassembly=``, with the 4-tuple as ``flow=``, it lets the completing - fragment's message decode, for the first record of each datagram only. - :class:`~pcapkit.foundation.extraction.Extractor` does not drive it - (:issue:`2496`). + completes (:issue:`2440`, :pr:`2451`). It is opt-in, as ``dtls=True`` with + ``reassembly=True`` to :class:`~pcapkit.foundation.extraction.Extractor` or + :func:`~pcapkit.interface.core.extract`, and runs on the default engine + alone, fed by :func:`~pcapkit.toolkit.pcap.dtls_reassembly` and its + :mod:`pcapkit.toolkit.pcapng` counterpart from the first record of each + datagram only; any other engine turns it off with a warning. + :func:`~pcapkit.foundation.analysis.dtls.dtls_handshake` decodes a + reassembled message (:issue:`2496`, :pr:`2528`). * **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an @@ -147,9 +163,40 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those - apply; a fragment of a larger packet, or a header it cannot walk, raises - :exc:`~pcapkit.utilities.exceptions.ProtocolError`. It is not a layer on the - wire. + apply; a fragment of a larger packet, or a header it cannot walk at any + depth of the chain, raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` + (:issue:`2513`, :pr:`2543`). It is not a layer on the wire. + +* **Added** -- verification and computation of the one's-complement Internet + checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, + in :mod:`pcapkit.protocols.misc.checksum.checksum` (:issue:`2470`, + :issue:`2472`, :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`). + Parsing verifies only under ``PCAPKIT_CHECKSUM_VERIFY``, off by default, + recording a :class:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus` + as ``checksum_status`` and warning on a mismatch with + :exc:`~pcapkit.utilities.warnings.ChecksumWarning`, a + :exc:`~pcapkit.utilities.warnings.ProtocolWarning` a filter can silence + alone. Construction computes a checksum the caller leaves out, unless + ``PCAPKIT_CHECKSUM_COMPUTE`` is off. An IPv4 or IPv6 layer verifies and + computes its upper layer's checksum over the + :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` it builds, and + ``make()`` of TCP, UDP and ICMPv6 also takes a ``pseudo``. What checksum + offload leaves on the sending host is ``NOT_COMPUTED`` rather than a + mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers + octets the fragment lacks, is ``NO_PSEUDO_HEADER``. + +* **Added** -- :func:`~pcapkit.foundation.registry.protocols.register_ipx`, + :func:`~pcapkit.foundation.registry.protocols.register_loopback`, + :func:`~pcapkit.foundation.registry.protocols.register_sctp_chunk`, + :func:`~pcapkit.foundation.registry.protocols.register_sctp_parameter` and + :func:`~pcapkit.foundation.registry.protocols.register_sctp_cause`, wrapping + IPX's packet-type map, Loopback's address-family map and SCTP's chunk, + chunk-parameter and error-cause parser maps with the validation the other + wrappers carry, and the + :meth:`IPX.register ` and + :meth:`Loopback.register ` + classmethods the IPX and Loopback wrappers call (:issue:`2511`, + :pr:`2521`). * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, @@ -166,11 +213,94 @@ pseudo-header, and stricter handling of a malformed IPv6 extension header. is :data:`None` unless a caller supplies one; the ``Raw`` layer's ``info.error`` holds the failure instead. +* **Changed** -- **a breaking change to** construction: a checksum left out of + ``make()`` is computed rather than zero, so the octets built change + (:issue:`2471`, :pr:`2543`). :class:`~pcapkit.protocols.internet.ipv4.IPv4` + computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP + payload built without one, and + :class:`~pcapkit.protocols.application.ospf.OSPF` that of an OSPFv2 packet. + A checksum given is written as given, and turning + ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. + +* **Changed** -- **a breaking change to** + :class:`~pcapkit.protocols.application.drarp.DRARP`: it moves from + :mod:`pcapkit.protocols.application.rarp` to + :mod:`pcapkit.protocols.application.drarp`, with no re-export at the old + path, as a module holds one protocol class (:issue:`2529`, :pr:`2540`). + ``pcapkit.DRARP``, ``pcapkit.protocols.DRARP`` and a parsed RARP's + ``DRARP`` alias are unchanged. + +* **Changed** -- **a breaking change to** IPv6 reassembly: a fragment + overlapping another of its datagram, other than an exact duplicate, abandons + the datagram, reported as + :attr:`Completion.OVERLAP `, + as :rfc:`8200#section-4.5` requires; 1.5.x overwrote the octets it overlapped, + as IPv4 still does under :rfc:`791` (:issue:`2547`, :pr:`2569`). + :class:`~pcapkit.foundation.reassembly.data.ip.Buffer` gains a required + ``extent`` field. + +* **Changed** -- **a breaking change to** TCP reassembly's gap cap: + ``pcapkit.foundation.reassembly.tcp.TCP.__window__`` is removed with no + alias, and the cap is :data:`~pcapkit.utilities.config.TCP_GAP_LIMIT`, set + by ``PCAPKIT_TCP_GAP_LIMIT``, 16 MiB by default as before and ``-1`` for no + limit (:issue:`2546`, :pr:`2579`). It is a safety limit, read each time + :meth:`TCP.reassembly ` + runs, and ``-1`` or a value above the default warns with + :exc:`~pcapkit.utilities.warnings.ConfigWarning`. + +* **Changed** -- **a breaking change to** + :class:`~pcapkit.foundation.extraction.Extractor` and + :func:`~pcapkit.interface.core.extract`: their ``dtls`` parameter, above, + follows ``tcp``, so a positional argument from ``buffer_size`` on moves one + place along, and :class:`~pcapkit.foundation.reassembly.ReassemblyManager` + takes a required ``dtls`` (:issue:`2496`, :pr:`2528`). + +* **Changed** -- protocol class bodies are reordered to the member-order + convention, with no change in behaviour, and a project test guards the + order (:issue:`2490`, :pr:`2522`, :pr:`2523`, :pr:`2538`, :pr:`2539`, + :pr:`2541`). + +* **Changed** -- the house conventions record that a protocol class takes its + module's name, one to a module (:issue:`2529`, :pr:`2542`); that a + docstring or prose block carries one ``versionadded`` per version, without + a body that only names the thing added (:issue:`2530`, :pr:`2542`); and that + each class-level registry map has a wrapper in + :mod:`pcapkit.foundation.registry` (:issue:`2511`, :pr:`2521`). ``const`` is + a label added alongside a commit type rather than a type, in the pull + request template and ``CONTRIBUTING.md`` too, and a span the transport + registries assign to a service keeps its bare name, the mint criterion's + exception to code-suffixing (:issue:`2549`, :issue:`2550`, :pr:`2570`). + +* **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker + per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers + on the 4-CPU runner; the weekly CI timing report reads the latest Python's + leg too (:issue:`2571`, :pr:`2572`). + * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, :pr:`2483`). +* **Fixed** -- :meth:`AppType.get ` + takes the ``default`` + :meth:`EnumLookup.get ` advertises, a + registered member's value that it never mints, and a miss raises + :exc:`~pcapkit.utilities.exceptions.EnumValueError`, still a + :exc:`ValueError`, where it raised a builtin one (:issue:`2545`, + :pr:`2568`). + +* **Fixed** -- under ``beholder``, a next layer captured at full length whose + parse raised, such as a malformed TCP header, reported its warnings before + being replaced with :class:`~pcapkit.protocols.misc.raw.Raw`. Its trial now + holds them however much was captured and drops them with the layer, so a + kept layer's warnings follow those of the layers it carries (:issue:`2516`, + :pr:`2519`). + +* **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of + a trial parse they rejected. The extension-header trial now drops them as + under ``beholder``, and the IPv4 and TCP header check holds its trial's and + discards them when it raises (:issue:`2526`, :pr:`2577`). + * **Fixed** -- selecting ``'IPv6_Ext'`` matched none of the dedicated extension headers: :meth:`IPv6_Ext.id() ` now From 521f75040fe4834710a89fb5ef3f0557eeabfec8 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 14:05:31 -0500 Subject: [PATCH 05/27] docs(changelog): fold HTTP dispatch, docs pages and registry conventions into 1.6.0 (#2449) - Breaking: HTTP dispatches to HTTPv1/HTTPv2 on read, make and from_data, and so does a transport payload, whose data key moves from http to httpv1/httpv2; the members and their schema and data classes are renamed from httpv1.HTTP/httpv2.HTTP with no re-export; id() takes the RFC names; unrecognised transport payloads are Raw with no error (#2544). - Changed: conventions record the dispatching base (#2544) and the case-fold exceptions (#2583); a project test guards every shipped module's documentation entry (#2584). Summary names the HTTP change. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 +++-- docs/source/changelog/1.6.0.rst | 56 +++++++++++++++++++++++++++++---- 2 files changed, 56 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7240a87efb..c840e26bbf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ ## 1.6.0 -- unreleased -Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, and stricter handling of a malformed IPv6 extension header. +Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, stricter handling of a malformed IPv6 extension header, and HTTP dispatched to its HTTP/1 and HTTP/2 classes under the RFC names. - **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). @@ -34,6 +34,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `DRARP`: it moves from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp`, with no re-export at the old path, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540)). `pcapkit.DRARP`, `pcapkit.protocols.DRARP` and a parsed RARP's `DRARP` alias are unchanged. +- **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. + - **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. - **Changed** -- **a breaking change to** TCP reassembly's gap cap: `pcapkit.foundation.reassembly.tcp.TCP.__window__` is removed with no alias, and the cap is `TCP_GAP_LIMIT`, set by `PCAPKIT_TCP_GAP_LIMIT`, 16 MiB by default as before and `-1` for no limit ([#2546](https://github.com/JarryShaw/PyPCAPKit/issues/2546), [#2579](https://github.com/JarryShaw/PyPCAPKit/pull/2579)). It is a safety limit, read each time `TCP.reassembly` runs, and `-1` or a value above the default warns with `ConfigWarning`. @@ -42,7 +44,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). + +- **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). - **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 8fde3ebe0f..bd41bf3277 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -5,8 +5,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each ``arphrd_type`` they do not read as an EtherType, the IP -pseudo-header, verified and computed one's-complement checksums, and stricter -handling of a malformed IPv6 extension header. +pseudo-header, verified and computed one's-complement checksums, stricter +handling of a malformed IPv6 extension header, and HTTP dispatched to its +HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the @@ -230,6 +231,34 @@ handling of a malformed IPv6 extension header. ``pcapkit.DRARP``, ``pcapkit.protocols.DRARP`` and a parsed RARP's ``DRARP`` alias are unchanged. +* **Changed** -- **a breaking change to** HTTP: parsing octets through + :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields + and its ``from_data`` give the + :class:`~pcapkit.protocols.application.httpv1.HTTPv1` or + :class:`~pcapkit.protocols.application.httpv2.HTTPv2` member, never an + instance of the base, and so does a TCP or UDP payload on an HTTP port, whose + data sits under ``httpv1`` or ``httpv2`` where it sat under ``http`` + (:issue:`2524`, :pr:`2544`). The members are renamed after their modules, + ``httpv1.HTTP`` and ``httpv2.HTTP`` becoming ``HTTPv1`` and ``HTTPv2`` along + with their schema and data classes, with no re-export at the old paths + (:issue:`2529`); ``pcapkit.HTTPv1``, ``pcapkit.HTTPv2`` and the package + exports of those names are unchanged. ``id()`` takes the RFC names, + ``('HTTP/1',)`` and ``('HTTP/2',)``, and the base's lists both, where they + were ``('HTTP', 'HTTPv1')``, ``('HTTP', 'HTTPv2')`` and + ``('HTTP', 'HTTPv1', 'HTTPv2')``. So a layer no longer compares equal to + ``'HTTP'``, ``'HTTPv1'`` or ``'HTTPv2'``; ``'HTTP'`` still selects either + version, and each version's own name, ``'HTTP/1'`` or ``'HTTPv1'`` say, + selects it alone, where any HTTP layer matched ``'HTTPv1'`` and ``'HTTPv2'`` + both. On the transport path a payload neither version recognises, a bare + connection preface included, is :class:`~pcapkit.protocols.misc.raw.Raw` + with no ``error``, where it carried ``'HTTP: unknown version'`` or + ``'HTTP/2: connection preface with no frame'``; a direct ``HTTP(...)`` on it + still raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`. One that + passes for an HTTP/2 frame header but ``HTTPv2`` refuses carries that + parser's own error on either path, where it was ``'HTTP: unknown version'``. + ``HTTPv2`` itself now parses a payload opening with the connection preface, + kept as ``preface``, and its ``make()`` takes ``preface``. + * **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as @@ -262,14 +291,29 @@ handling of a malformed IPv6 extension header. * **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module (:issue:`2529`, :pr:`2542`); that a - docstring or prose block carries one ``versionadded`` per version, without - a body that only names the thing added (:issue:`2530`, :pr:`2542`); and that - each class-level registry map has a wrapper in + versioned family's base dispatches on every path, so parsing, building and + ``from_data`` give a member rather than the base (:issue:`2524`, + :pr:`2544`); that a docstring or prose block carries one ``versionadded`` + per version, without a body that only names the thing added (:issue:`2530`, + :pr:`2542`); and that each class-level registry map has a wrapper in :mod:`pcapkit.foundation.registry` (:issue:`2511`, :pr:`2521`). ``const`` is a label added alongside a commit type rather than a type, in the pull request template and ``CONTRIBUTING.md`` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's - exception to code-suffixing (:issue:`2549`, :issue:`2550`, :pr:`2570`). + exception to code-suffixing (:issue:`2549`, :issue:`2550`, :pr:`2570`). The + registry conventions record the exceptions to their case-folding rules -- + the crawlers that re-case the names they generate, the ``Method`` + constructor's fold, and ``Command.get`` and ``Method.get`` answering a name + miss with an unregistered member -- and the folds that live outside any + registry (:issue:`2551`, :issue:`2552`, :issue:`2553`, :issue:`2556`, + :issue:`2565`, :issue:`2582`, :pr:`2583`). + +* **Changed** -- a project test fails on a shipped module with no + documentation entry, the placeholder sources under ``NotImplemented/`` + excepted, and the modules this release adds that had none -- + :mod:`pcapkit.const.linux`, the :mod:`pcapkit.vendor.tls` and + :mod:`pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have + one (:issue:`2559`, :pr:`2584`). * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers From 42280b49753a5512819f983a1c32e2a3a639c774 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 21:25:08 -0500 Subject: [PATCH 06/27] docs(changelog): fold DCCP, InARP's move and main's latest wave into 1.6.0 (#2449) - Added: DCCP, its checksum over the IP pseudo-header, register_dccp and register_dccp_option, and the pcapkit.const.dccp registries (#2593). - Folded into unreleased entries: LinuxSLL's stop on its ARPHRD path (#2587) and ARPHRD's __proto_dest__ (#2598), SSLv2 records chained in a segment (#2585), the checksum's move to pcapkit.corekit.checksum.internet (#2586), and IGMPv2's and IGMPv3's versioned name (#2590). - Breaking: InARP moves to pcapkit.protocols.link.inarp with no re-export, joining the DRARP entry (#2596). - Changed: the conventions' new rulings (#2599), the milestone workflow's next-open-milestone lookup (#2591), the PyPy protocols shards (#2604). Fixed: a handled next-layer refusal is quiet under beholder (#2597); the IPX crawlers restore sys.path (#2601). - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 22 +++++--- docs/source/changelog/1.6.0.rst | 96 +++++++++++++++++++++++++++------ 2 files changed, 93 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c840e26bbf..9cceab7760 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, stricter handling of a malformed IPv6 extension header, and HTTP dispatched to its HTTP/1 and HTTP/2 classes under the RFC names. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)), as does a protocol class declaring `code=ARPHRD.X`, since `ARPHRD` names `LinuxSLL` as its `__proto_dest__` ([#2533](https://github.com/JarryShaw/PyPCAPKit/issues/2533), [#2598](https://github.com/JarryShaw/PyPCAPKit/pull/2598)). A parse told to stop there -- at `LinuxSLL`, the version itself or the link layer -- keeps the payload `Raw` on either path ([#2532](https://github.com/JarryShaw/PyPCAPKit/issues/2532), [#2587](https://github.com/JarryShaw/PyPCAPKit/pull/2587)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). - **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). @@ -14,17 +14,19 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. + +- **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. - **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). -- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. +- **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. - **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.protocols.misc.checksum.checksum` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). @@ -32,7 +34,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. -- **Changed** -- **a breaking change to** `DRARP`: it moves from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp`, with no re-export at the old path, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540)). `pcapkit.DRARP`, `pcapkit.protocols.DRARP` and a parsed RARP's `DRARP` alias are unchanged. +- **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. @@ -44,17 +46,19 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). + +- **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). - **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). -- **Fixed** -- under `beholder`, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before being replaced with `Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries ([#2516](https://github.com/JarryShaw/PyPCAPKit/issues/2516), [#2519](https://github.com/JarryShaw/PyPCAPKit/pull/2519)). +- **Fixed** -- under `beholder`, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before being replaced with `Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries ([#2516](https://github.com/JarryShaw/PyPCAPKit/issues/2516), [#2519](https://github.com/JarryShaw/PyPCAPKit/pull/2519)). A `ProtocolError` built while `beholder` runs a next layer is now quiet, so a refusal kept as `Raw` is recorded on that layer's `error` alone, where a loud one logged at `CRITICAL` and installed the exception hooks, and `beholder` logged it again at `WARNING`; building the refused layer directly stays loud ([#2588](https://github.com/JarryShaw/PyPCAPKit/issues/2588), [#2597](https://github.com/JarryShaw/PyPCAPKit/pull/2597)). - **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as under `beholder`, and the IPv4 and TCP header check holds its trial's and discards them when it raises ([#2526](https://github.com/JarryShaw/PyPCAPKit/issues/2526), [#2577](https://github.com/JarryShaw/PyPCAPKit/pull/2577)). @@ -62,6 +66,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). +- **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). + --- Full changelog: diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index bd41bf3277..bcf369df32 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -22,7 +22,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. each of which has a dissector of its own, below (:issue:`2460`), reached through ``LinuxSLL``'s own registry keyed by ``arphrd_type``, which :func:`~pcapkit.foundation.registry.protocols.register_arphrd` extends - (:issue:`2510`, :pr:`2527`). The hardware types are + (:issue:`2510`, :pr:`2527`), as does a protocol class declaring + ``code=ARPHRD.X``, since ``ARPHRD`` names ``LinuxSLL`` as its + ``__proto_dest__`` (:issue:`2533`, :pr:`2598`). A parse told to stop there -- at ``LinuxSLL``, + the version itself or the link layer -- keeps the payload + :class:`~pcapkit.protocols.misc.raw.Raw` on either path (:issue:`2532`, + :pr:`2587`). The hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the ```` values that table lists, so any other, @@ -106,9 +111,29 @@ HTTP/1 and HTTP/2 classes under the RFC names. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and ``('IGMPv3',)``; the base's is ``('IGMP', 'IGMPv2', 'IGMPv3')``, each - member's canonical name (:issue:`2512`, :pr:`2506`, :pr:`2518`). The - registries are :mod:`pcapkit.const.igmp`: the types crawled from IANA, the - group record types from a hand-maintained table. + member's canonical name (:issue:`2512`, :pr:`2506`, :pr:`2518`). IGMPv2's + and IGMPv3's ``name`` carries the version, + ``'Internet Group Management Protocol version 2'`` say, while IGMPv1's, as + in :rfc:`1112`, does not (:issue:`2558`, :pr:`2590`). The registries are + :mod:`pcapkit.const.igmp`: the types crawled from IANA, the group record + types from a hand-maintained table. + +* **Added** -- :class:`~pcapkit.protocols.transport.dccp.DCCP` (:rfc:`4340`) + for IP protocol 33, which was kept as + :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2489`, :pr:`2593`). It + reads and builds the generic header with short and long sequence numbers, + the fields of every packet type, DCCP-Listen (:rfc:`5596`) and the options; + the Data Checksum option's CRC-32c is recorded rather than checked. Its + checksum is verified and computed as TCP's is, below, over the enclosing IP + layer's pseudo-header and the part of the packet its Checksum Coverage + gives (:rfc:`4340#section-9.2`): a coverage running past the packet is + ``INVALID`` and warns, and a ``make()`` that computes the checksum writes a + coverage reaching the end of the packet as 0, as Linux did. + :func:`~pcapkit.foundation.registry.protocols.register_dccp` registers a + port's dissector, and + :func:`~pcapkit.foundation.registry.protocols.register_dccp_option` an + option parser. The registries are :mod:`pcapkit.const.dccp`, crawled from + IANA. * **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` @@ -141,8 +166,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays :class:`~pcapkit.protocols.misc.raw.Raw`; a kept one drops the rejected TLS trial's warnings (:issue:`2525`, - :pr:`2537`). Its constants are :mod:`pcapkit.const.sslv2`, from a - hand-maintained table. + :pr:`2537`). Each whole record after the first in a TCP segment, whatever + its message type, is a further SSLv2 layer, the previous record's payload, + as with TLS -- a ``CLIENT-HELLO`` then a ``CLIENT-MASTER-KEY`` read + ``SSLv2:SSLv2`` -- and one that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment + (:issue:`2567`, :pr:`2585`). Its constants are :mod:`pcapkit.const.sslv2`, + from a hand-maintained table. * **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented @@ -170,10 +200,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- verification and computation of the one's-complement Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, - in :mod:`pcapkit.protocols.misc.checksum.checksum` (:issue:`2470`, - :issue:`2472`, :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`). + in :mod:`pcapkit.corekit.checksum.internet` (:issue:`2470`, :issue:`2472`, + :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`), where + :mod:`pcapkit.corekit.checksum` holds a module per checksum family and + imports nothing from :mod:`pcapkit.protocols` (:issue:`2580`, :pr:`2586`). Parsing verifies only under ``PCAPKIT_CHECKSUM_VERIFY``, off by default, - recording a :class:`~pcapkit.protocols.misc.checksum.checksum.ChecksumStatus` + recording a :class:`~pcapkit.corekit.checksum.internet.ChecksumStatus` as ``checksum_status`` and warning on a mismatch with :exc:`~pcapkit.utilities.warnings.ChecksumWarning`, a :exc:`~pcapkit.utilities.warnings.ProtocolWarning` a filter can silence @@ -224,12 +256,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. * **Changed** -- **a breaking change to** - :class:`~pcapkit.protocols.application.drarp.DRARP`: it moves from + :class:`~pcapkit.protocols.application.drarp.DRARP` and + :class:`~pcapkit.protocols.link.inarp.InARP`, which move from :mod:`pcapkit.protocols.application.rarp` to - :mod:`pcapkit.protocols.application.drarp`, with no re-export at the old - path, as a module holds one protocol class (:issue:`2529`, :pr:`2540`). - ``pcapkit.DRARP``, ``pcapkit.protocols.DRARP`` and a parsed RARP's - ``DRARP`` alias are unchanged. + :mod:`pcapkit.protocols.application.drarp` and from + :mod:`pcapkit.protocols.link.arp` to :mod:`pcapkit.protocols.link.inarp`, + with no re-export at the old paths, as a module holds one protocol class + (:issue:`2529`, :pr:`2540`, :pr:`2596`). ``pcapkit.DRARP``, + ``pcapkit.InARP``, their :mod:`pcapkit.protocols` exports and a parsed + RARP's ``DRARP`` or ARP's ``InARP`` alias are unchanged. * **Changed** -- **a breaking change to** HTTP: parsing octets through :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields @@ -306,7 +341,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. constructor's fold, and ``Command.get`` and ``Method.get`` answering a name miss with an unregistered member -- and the folds that live outside any registry (:issue:`2551`, :issue:`2552`, :issue:`2553`, :issue:`2556`, - :issue:`2565`, :issue:`2582`, :pr:`2583`). + :issue:`2565`, :issue:`2582`, :pr:`2583`). They also record the SSL + family's grouping, and that a family base is bound, and dispatches, only + where the registry code cannot tell its members apart; that a pseudo-header + protocol belongs in :mod:`pcapkit.protocols.misc`; that + :mod:`pcapkit.utilities.compat` holds compatibility implementations and + re-exports only, behind version gates; and how a ruling given in a pull + request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -318,7 +359,17 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's - leg too (:issue:`2571`, :pr:`2572`). + leg too (:issue:`2571`, :pr:`2572`). A PyPy leg runs + ``tests/protocols/internet``, ``tests/protocols/application`` and + ``tests/protocols/misc`` as shards of their own ahead of the rest of + ``tests/protocols``, which run whole outgrew the runner's memory and + stalled at its tail (:issue:`2600`, :pr:`2604`). + +* **Changed** -- once a release's milestone is closed while ``__version__`` + still names that release, the milestone workflow gives an item the lowest + open release milestone above it, compared as numbers so ``v1.10`` follows + ``v1.9``, and a backfill starts above the closed milestone's items + (:issue:`2548`, :pr:`2591`). * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and @@ -338,7 +389,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. being replaced with :class:`~pcapkit.protocols.misc.raw.Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries (:issue:`2516`, - :pr:`2519`). + :pr:`2519`). A :exc:`~pcapkit.utilities.exceptions.ProtocolError` built + while ``beholder`` runs a next layer is now quiet, so a refusal kept as + ``Raw`` is recorded on that layer's ``error`` alone, where a loud one logged + at ``CRITICAL`` and installed the exception hooks, and ``beholder`` logged + it again at ``WARNING``; building the refused layer directly stays loud + (:issue:`2588`, :pr:`2597`). * **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as @@ -358,3 +414,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. The layer now holds :class:`~pcapkit.protocols.misc.null.NoPayload`, the upper layer belonging to the enclosing IPv6 header (:issue:`2484`, :issue:`2487`, :pr:`2486`, :pr:`2488`). + +* **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or + socket crawler, without ``requests`` installed lost an entry of + :data:`sys.path` on each failed attempt, so a :func:`pkgutil.walk_packages` + retrying it drained :data:`sys.path` until it raised :exc:`IndexError`. The + entry is now restored however the import ends (:issue:`2589`, :pr:`2601`). From 5ac9441beec98b39ebeed4064e6428b1f65d0eb2 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 22:10:06 -0500 Subject: [PATCH 07/27] docs(changelog): fold HTTP/2 frame chaining, SSLv3's module and main's latest wave into 1.6.0 (#2449) - Added: every whole HTTP/2 frame in a TCP segment is a further HTTPv2 layer, the previous frame's payload, as with TLS (#2609). - Folded into unreleased entries: SSLv3 lives in pcapkit.protocols.application.sslv3 (#2602), and a reassembled DTLS Datagram's packet holds the deferred handshake decode (#2605). tls.SSLv3 is in no release tag, so the entry records no move. - Changed: the remaining xdist jobs at -n logical (#2606) join the CI entry, and the member-order entry cites the last reorder (#2613). - No entry for the test-only #2611 and #2612, or for #2610's private registry sentinel, which changes no behaviour. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 10 ++++++---- docs/source/changelog/1.6.0.rst | 29 +++++++++++++++++++++++++---- 2 files changed, 31 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9cceab7760..0f8b4bf26d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,11 +18,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3` exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. -- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). +- **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each after the first, which was kept as `Raw`, is a further `HTTPv2` layer, the previous frame's payload, as with TLS -- a `SETTINGS` then a `WINDOW_UPDATE` frame read `TCP:HTTP/2:HTTP/2` ([#2536](https://github.com/JarryShaw/PyPCAPKit/issues/2536), [#2609](https://github.com/JarryShaw/PyPCAPKit/pull/2609)). One that fails to decode is `Raw` with the rest of the segment and the parser's `error`, and a frame the segment ends inside stays `Raw` with none. A parse told to stop at HTTP/2 keeps the rest of the segment raw, and `FRAME_LAYER_LIMIT` bounds how many frames are dissected. + +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. @@ -44,13 +46,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). -- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541)). +- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). - **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index bcf369df32..bd9261a003 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -150,7 +150,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment - (:issue:`2517`, :pr:`2537`). :class:`~pcapkit.protocols.application.tls.SSLv3` + (:issue:`2517`, :pr:`2537`). + :class:`~pcapkit.protocols.application.sslv3.SSLv3`, in + :mod:`pcapkit.protocols.application.sslv3` (:issue:`2529`, :pr:`2602`), exists for explicit construction only. A record split across TCP segments is not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also reads and builds DTLS 1.3's unified-header records @@ -174,6 +176,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. (:issue:`2567`, :pr:`2585`). Its constants are :mod:`pcapkit.const.sslv2`, from a hand-maintained table. +* **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each + after the first, which was kept as :class:`~pcapkit.protocols.misc.raw.Raw`, + is a further :class:`~pcapkit.protocols.application.httpv2.HTTPv2` layer, the + previous frame's payload, as with TLS -- a ``SETTINGS`` then a + ``WINDOW_UPDATE`` frame read ``TCP:HTTP/2:HTTP/2`` (:issue:`2536`, + :pr:`2609`). One that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment and + the parser's ``error``, and a frame the segment ends inside stays + :class:`~pcapkit.protocols.misc.raw.Raw` with none. A parse told to stop at + HTTP/2 keeps the rest of the segment raw, and + :data:`~pcapkit.utilities.config.FRAME_LAYER_LIMIT` bounds how many frames + are dissected. + * **Added** -- :class:`pcapkit.foundation.reassembly.dtls.DTLS`, exported as ``DTLS_Reassembly``, which reassembles a DTLS handshake message fragmented across records (:rfc:`6347#section-4.2.3`), keyed by the UDP 4-tuple and the @@ -185,7 +200,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. :func:`~pcapkit.foundation.analysis.dtls.dtls_handshake` decodes a - reassembled message (:issue:`2496`, :pr:`2528`). + reassembled message (:issue:`2496`, :pr:`2528`). A reassembled + :class:`Datagram ` holds + that decode as ``packet``, run on its first read as an IP or TCP datagram's + is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). * **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an @@ -322,7 +340,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order (:issue:`2490`, :pr:`2522`, :pr:`2523`, :pr:`2538`, :pr:`2539`, - :pr:`2541`). + :pr:`2541`, :pr:`2613`). * **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module (:issue:`2529`, :pr:`2542`); that a @@ -359,7 +377,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's - leg too (:issue:`2571`, :pr:`2572`). A PyPy leg runs + leg too (:issue:`2571`, :pr:`2572`). The integration, engine, parity, + project-test and gate jobs and the experimental workflow's Windows and + free-threaded 3.14t legs run ``-n logical`` too, and a project test fails + on a bare ``-n auto`` (:issue:`2573`, :pr:`2606`). A PyPy leg runs ``tests/protocols/internet``, ``tests/protocols/application`` and ``tests/protocols/misc`` as shards of their own ahead of the rest of ``tests/protocols``, which run whole outgrew the runner's memory and From ec47f7f552ebc58e3730046e813a366a489d7153 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 22:24:34 -0500 Subject: [PATCH 08/27] docs(changelog): fold TLS extensions and DTLS record chaining into 1.6.0 (#2449) - Folded into the TLS/DTLS entry: the hello and EncryptedExtensions extension lists, their registered body decoders, register_tls_extension and the pcapkit.const.tls extension types (#2595), and each whole DTLS record in a datagram as a further layer (#2603). - The conventions entry records #2611's per-test purge rule; the member-order entry cites #2491. - No entry for #2595's private ProtocolBase value helpers, as for the other private API. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 6 +++--- docs/source/changelog/1.6.0.rst | 32 ++++++++++++++++++++++++-------- 2 files changed, 27 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f8b4bf26d..4e4fdb7e70 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extension types, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). The content and handshake types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. @@ -46,9 +46,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). -- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). +- **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index bd9261a003..1cd6025c62 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -144,9 +144,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. operation, telling 1.3 apart by its ``supported_versions`` extension; ``id()`` stays ``('TLS',)`` or ``('DTLS',)``. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, - compression methods and extension types, any other message at type and - length -- as are an alert and ChangeCipherSpec, while encrypted records stay - opaque octets. Each whole TLS record in a TCP segment is a layer of its own, + compression methods and extensions, EncryptedExtensions down to its + extensions, any other message at type and length -- as are an alert and + ChangeCipherSpec, while encrypted records stay opaque octets. An extension's + body is read and built for ``server_name``, ``supported_groups``, + ``signature_algorithms``, ALPN, ``supported_versions`` and ``key_share``, a + HelloRetryRequest's included, and kept as octets for any other type or for a + body its parser rejects, which logs nothing; + :func:`~pcapkit.foundation.registry.protocols.register_tls_extension` + registers a parser and constructor for a further type (:issue:`2514`, + :pr:`2595`). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment @@ -158,8 +165,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. reads and builds DTLS 1.3's unified-header records (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, - :pr:`2494`). The content and handshake types are :mod:`pcapkit.const.tls`, - crawled from IANA (:issue:`2495`, :pr:`2503`). + :pr:`2494`). Each whole DTLS record in a UDP datagram is likewise a layer of + its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read + ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to + the end of the datagram, and one that fails to decode is + :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the datagram + (:issue:`2535`, :pr:`2603`). The content, handshake and extension types are + :mod:`pcapkit.const.tls`, crawled from IANA (:issue:`2495`, :issue:`2514`, + :pr:`2503`, :pr:`2595`). * **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares @@ -339,8 +352,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the - order (:issue:`2490`, :pr:`2522`, :pr:`2523`, :pr:`2538`, :pr:`2539`, - :pr:`2541`, :pr:`2613`). + order (:issue:`2490`, :pr:`2491`, :pr:`2522`, :pr:`2523`, :pr:`2538`, + :pr:`2539`, :pr:`2541`, :pr:`2613`). * **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module (:issue:`2529`, :pr:`2542`); that a @@ -365,7 +378,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. protocol belongs in :mod:`pcapkit.protocols.misc`; that :mod:`pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull - request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). + request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). The + test conventions record that a test class whose tests each need a fresh + import purges in ``setUp``, with a comment saying why, rather than importing + once per class (:issue:`2562`, :pr:`2611`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` From 34f32f4f7f9d8ac06cc60d981274967dfd1e0e9d Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 23:05:33 -0500 Subject: [PATCH 09/27] docs(changelog): fold IEEE 802.11 and the IP/TCP analysis functions into 1.6.0 (#2449) - Added: IEEE802_11, the 802.11 MAC header behind radiotap, with the pcapkit.const.ieee802_11 type and subtype registries (#2607); the Radiotap entry drops its stale "no dissector yet" note and names its own registry and register_radiotap. - Added: ip_datagram and tcp_datagram beside dtls_handshake (#2619). - The conventions entry records #2607's ruling on a paywalled registry's pinned secondary source. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 ++++++-- docs/source/changelog/1.6.0.rst | 33 ++++++++++++++++++++++++++++++--- 2 files changed, 36 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e4fdb7e70..358b12921a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded, and the 802.11 MAC frame behind them, which has no dissector in this package yet, as `Raw`. +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). + +- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. @@ -26,6 +28,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). +- **Added** -- `ip_datagram()` and `tcp_datagram()`, siblings of `dtls_handshake`, above, which parse the payload of a datagram that IPv4, IPv6 or TCP reassembly reports, by its payload protocol or by its port pair, and return `None` for the tuple of received runs `strict=True` reports for an incomplete one ([#2581](https://github.com/JarryShaw/PyPCAPKit/issues/2581), [#2619](https://github.com/JarryShaw/PyPCAPKit/pull/2619)). + - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. - **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. @@ -48,7 +52,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 1cd6025c62..32b1237002 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -78,8 +78,25 @@ HTTP/1 and HTTP/2 classes under the RFC names. It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than - decoded, and the 802.11 MAC frame behind them, which has no dissector in this - package yet, as :class:`~pcapkit.protocols.misc.raw.Raw`. + decoded. The 802.11 MAC frame behind them is dissected as + :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, below, looked up in + ``Radiotap``'s own registry under + :attr:`LinkType.IEEE802_11 `, + and :func:`~pcapkit.foundation.registry.protocols.register_radiotap` + replaces that dissector (:issue:`2575`, :pr:`2607`). + +* **Added** -- :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, a + dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, + which :class:`~pcapkit.protocols.link.radiotap.Radiotap` dispatches to + (:issue:`2575`, :pr:`2607`). It reads Frame Control, the Control Frame + Extension and S1G Beacon layouts included, and the address, sequence, QoS + Control and HT Control fields each frame type carries; a Control frame of a + subtype it does not model, and an Extension frame, are read as the minimal + header. The frame body is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, + with the FCS and radiotap's pad octets where captured, as ``Radiotap`` does + not decode the Flags that mark them, and any other protocol version is + refused. The frame types and subtypes are :mod:`pcapkit.const.ieee802_11`, + from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. * **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` (:rfc:`792`) and :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` (:rfc:`4443`), on an @@ -218,6 +235,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. that decode as ``packet``, run on its first read as an IP or TCP datagram's is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). +* **Added** -- :func:`~pcapkit.foundation.analysis.ip.ip_datagram` and + :func:`~pcapkit.foundation.analysis.tcp.tcp_datagram`, siblings of + ``dtls_handshake``, above, which parse the payload of a datagram that IPv4, + IPv6 or TCP reassembly reports, by its payload protocol or by its port + pair, and return :data:`None` for the tuple of received runs ``strict=True`` + reports for an incomplete one (:issue:`2581`, :pr:`2619`). + * **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit @@ -381,7 +405,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. request's review is cited (:issue:`2557`, :issue:`2563`, :pr:`2599`). The test conventions record that a test class whose tests each need a fresh import purges in ``setUp``, with a comment saying why, rather than importing - once per class (:issue:`2562`, :pr:`2611`). + once per class (:issue:`2562`, :pr:`2611`). Where a registry's primary source + is paywalled, the const-vendor pairing accepts a pinned secondary revision as + a crawler's ``DATA`` once its values are cross-checked against independent + implementations (:issue:`2575`, :pr:`2607`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` From 52c6ec3930e285734084296daf668a679cd595e1 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Sun, 1 Nov 2026 23:40:14 -0500 Subject: [PATCH 10/27] docs(changelog): fold pad_length, the shared contextvars and the role kinds into 1.6.0 (#2449) - Added: pad_length and PaddingField's align/after form (#2624), and pcapkit.utilities.contextvars (#2623); the old private names get no entry, as for the other private API. - Changed: the weekly Issue and PR Kinds workflow, beside the milestone workflow's entry (#2626). - The conventions entry records where a context variable lives (#2623), and the role-kind citation rule with its test (#2626). - No entry for #2626's heading and Mermaid fixes, #2564 or fa9af520f. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 +++++++- docs/source/changelog/1.6.0.rst | 26 +++++++++++++++++++++++++- 2 files changed, 32 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 358b12921a..8f751e2b97 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -36,6 +36,10 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). +- **Added** -- `pcapkit.corekit.fields.strings.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)). + +- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `in_fragment` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). + - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. @@ -52,7 +56,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). @@ -60,6 +64,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). +- **Changed** -- a weekly Issue and PR Kinds workflow regenerates, with `util/issue_pr_kinds.py`, the snapshot of the tracker that the project test of each role's kind reads, and opens or updates a pull request from `bot/issue-pr-kinds` when it changes, never pushing `main` ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). + - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). - **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 32b1237002..2b95fc5812 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -286,6 +286,18 @@ HTTP/1 and HTTP/2 classes under the RFC names. classmethods the IPX and Loopback wrappers call (:issue:`2511`, :pr:`2521`). +* **Added** -- :func:`pcapkit.corekit.fields.strings.pad_length`, the octets of + padding that take a length to an ``align``-octet boundary, and a + :class:`~pcapkit.corekit.fields.strings.PaddingField` sized by it, given + ``align`` and ``after`` -- the boundary, and the field or callable whose + length the padding follows -- in place of ``length`` (:issue:`2614`, + :pr:`2624`). + +* **Added** -- :mod:`pcapkit.utilities.contextvars`, holding the context + variables more than one module sets or reads -- ``beholding``, + ``frame_depth``, ``in_fragment`` and ``rebuilding`` -- and importing nothing + from :mod:`pcapkit` (:issue:`2616`, :pr:`2623`). + * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as :class:`~pcapkit.protocols.misc.raw.Raw`, and the walk stops @@ -408,7 +420,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. once per class (:issue:`2562`, :pr:`2611`). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's ``DATA`` once its values are cross-checked against independent - implementations (:issue:`2575`, :pr:`2607`). + implementations (:issue:`2575`, :pr:`2607`). A context variable more than one + module uses lives in :mod:`pcapkit.utilities.contextvars`, and one that only + its own module uses stays private there (:issue:`2616`, :pr:`2623`). Each + ``:issue:`` and ``:pr:`` role in the documentation names a number of its own + kind, a ruling given in a pull request's review cited by the issue that pull + request closes, and a project test checks each role's kind against a + snapshot of the tracker (:issue:`2561`, :pr:`2626`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -435,6 +453,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``v1.9``, and a backfill starts above the closed milestone's items (:issue:`2548`, :pr:`2591`). +* **Changed** -- a weekly Issue and PR Kinds workflow regenerates, with + ``util/issue_pr_kinds.py``, the snapshot of the tracker that the project test + of each role's kind reads, and opens or updates a pull request from + ``bot/issue-pr-kinds`` when it changes, never pushing ``main`` + (:issue:`2561`, :pr:`2626`). + * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, From 7be441345a96327c80758a1deb5c4c84532be0ae Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 00:57:31 -0500 Subject: [PATCH 11/27] docs(changelog): fold Quick-Start's refusals, DTLS CIDs, 802.11 link types and the registry split into 1.6.0 (#2449) - Breaking: Quick-Start values their fields cannot carry raise, reversing 1.5.0's round-down of a TCP rate under 80 kbps (#2630). - Folded into unreleased entries: tls12_cid records and DTLSContext (#2631), LinkType 105 and 127 in Frame and PCAP-NG (#2632), and DTLS.analyze_handshake behind dtls_handshake and packet (#2634). - Changed: foundation.registry.protocols split into per-layer submodules (#2635); 1.6.0's register_* roles and 1.5.0's three register_protocol* roles point at their submodules, 1.5.0's text unchanged. - The conventions entry records the per-layer wrapper placement (#2635) and parse ownership (#2634). No entry for #2629, which changes no released behaviour. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 14 ++++-- docs/source/changelog/1.5.0.rst | 12 ++--- docs/source/changelog/1.6.0.rst | 86 +++++++++++++++++++++++++-------- 3 files changed, 82 insertions(+), 30 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f751e2b97..8641183f4d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,9 +10,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). -- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. +- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11` (105) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. @@ -20,13 +20,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw as an [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` record does ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)), and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves a `tls12_cid` record's connection ID from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. - **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each after the first, which was kept as `Raw`, is a further `HTTPv2` layer, the previous frame's payload, as with TLS -- a `SETTINGS` then a `WINDOW_UPDATE` frame read `TCP:HTTP/2:HTTP/2` ([#2536](https://github.com/JarryShaw/PyPCAPKit/issues/2536), [#2609](https://github.com/JarryShaw/PyPCAPKit/pull/2609)). One that fails to decode is `Raw` with the rest of the segment and the parser's `error`, and a frame the segment ends inside stays `Raw` with none. A parse told to stop at HTTP/2 keeps the rest of the segment raw, and `FRAME_LAYER_LIMIT` bounds how many frames are dissected. -- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). +- **Added** -- `pcapkit.foundation.reassembly.dtls.DTLS`, exported as `DTLS_Reassembly`, which reassembles a DTLS handshake message fragmented across records ([RFC 6347 Section 4.2.3](https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.3)), keyed by the UDP 4-tuple and the epoch, accepting overlapping fragments and holding a buffer until it completes ([#2440](https://github.com/JarryShaw/PyPCAPKit/issues/2440), [#2451](https://github.com/JarryShaw/PyPCAPKit/pull/2451)). It is opt-in, as `dtls=True` with `reassembly=True` to `Extractor` or `extract()`, and runs on the default engine alone, fed by `dtls_reassembly()` and its `pcapkit.toolkit.pcapng` counterpart from the first record of each datagram only; any other engine turns it off with a warning. `dtls_handshake()` decodes a reassembled message ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). A reassembled `Datagram` holds that decode as `packet`, run on its first read as an IP or TCP datagram's is, and `None` for an incomplete message ([#2531](https://github.com/JarryShaw/PyPCAPKit/issues/2531), [#2605](https://github.com/JarryShaw/PyPCAPKit/pull/2605)). Both run the classmethod `DTLS.analyze_handshake` -- `packet` through the reassembly class's own `__protocol_type__` -- which keeps a body that does not decode as raw octets, logging nothing ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). - **Added** -- `ip_datagram()` and `tcp_datagram()`, siblings of `dtls_handshake`, above, which parse the payload of a datagram that IPv4, IPv6 or TCP reassembly reports, by its payload protocol or by its port pair, and return `None` for the tuple of received runs `strict=True` reports for an incomplete one ([#2581](https://github.com/JarryShaw/PyPCAPKit/issues/2581), [#2619](https://github.com/JarryShaw/PyPCAPKit/pull/2619)). @@ -44,6 +44,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. +- **Changed** -- **a breaking change to** building a Quick-Start option in `IPv4`, `HOPOPT`, `IPv6_Opts` and `TCP`: a value its field cannot carry raises `ProtocolError`, where 1.5.x built it ([#2617](https://github.com/JarryShaw/PyPCAPKit/issues/2617), [#2630](https://github.com/JarryShaw/PyPCAPKit/pull/2630)). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and `True` as a TTL, nonce or reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. + - **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. @@ -54,9 +56,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `Extractor` and `extract()`: their `dtls` parameter, above, follows `tcp`, so a positional argument from `buffer_size` on moves one place along, and `ReassemblyManager` takes a required `dtls` ([#2496](https://github.com/JarryShaw/PyPCAPKit/issues/2496), [#2528](https://github.com/JarryShaw/PyPCAPKit/pull/2528)). +- **Changed** -- `pcapkit.foundation.registry.protocols` is a package with a submodule per layer -- `link`, `internet`, `transport`, `application` and `misc`, with the shared registrars in `protocol` -- and exports every public function they define, so each `register_*` imports from it and from `pcapkit.foundation.registry` as before ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A registrar's `__module__` names its submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. + - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index 9eeab1455e..c8b7d45ea0 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -1143,8 +1143,8 @@ Fixed ``register_reassembly`` and ``register_traceflow`` on :class:`.Extractor`, :meth:`.TraceFlow.register_dumper` (reached through ``register_traceflow_dumper``), and - :func:`pcapkit.foundation.registry.protocols.register_protocol`. Each guard gains an - explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that + :func:`pcapkit.foundation.registry.protocols.register_protocol `. + Each guard gains an explicit ``isinstance(x, type)`` test ahead of its ``issubclass``; in those that accept a :class:`.ModuleDescriptor` it runs after the descriptor is unwrapped, so a descriptor naming a non-class attribute is rejected too, while ``register_protocol`` takes no descriptor. No guard's target class changes, and a wrong class still raises @@ -1239,8 +1239,8 @@ Fixed and ``register_transtype`` refuse a ``code`` that is not an enum member with :exc:`.RegistryError` before touching a registry, where they registered it and then raised ``AttributeError``, leaving the entry behind (:issue:`2186`, :pr:`2193`). -* :func:`pcapkit.foundation.registry.protocols.register_protocol_code`, and the - ``code=`` class keyword, check every destination's key before registering any of +* :func:`pcapkit.foundation.registry.protocols.register_protocol_code `, + and the ``code=`` class keyword, check every destination's key before registering any of them. A key belonging to another enum, such as ``{Frame: TransType.ICMP}``, is refused with :exc:`.RegistryError` where it replaced the shipped ``LinkType.ETHERNET`` entry, and a bad target late in the list no longer leaves @@ -1382,8 +1382,8 @@ Added Inference refuses rather than guesses: an enum member whose type names no known destination raises :exc:`.RegistryError`, and an unrecognised class keyword raises :exc:`.UnsupportedCall`, matching the other registrar families. Backed by - :func:`pcapkit.foundation.registry.protocols.register_protocol_code`, also callable - directly for a class that declined at definition time. It was written up as the + :func:`pcapkit.foundation.registry.protocols.register_protocol_code `, + also callable directly for a class that declined at definition time. It was written up as the mechanism :issue:`548` needs; :issue:`548` turned out to need a missing class instead (115 is :rfc:`3931` L2TPv3 over IP, not :rfc:`2661`; see the corresponding **Fixed** entry below), so its worked example names ``L2TPv3`` (:pr:`570`). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 2b95fc5812..aa4bffa393 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -21,7 +21,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``FRAD``, ``NETLINK``, ``IPGRE``/``IP6GRE`` and ``IEEE80211_RADIOTAP`` -- each of which has a dissector of its own, below (:issue:`2460`), reached through ``LinuxSLL``'s own registry keyed by ``arphrd_type``, which - :func:`~pcapkit.foundation.registry.protocols.register_arphrd` extends + :func:`~pcapkit.foundation.registry.protocols.link.register_arphrd` extends (:issue:`2510`, :pr:`2527`), as does a protocol class declaring ``code=ARPHRD.X``, since ``ARPHRD`` names ``LinuxSLL`` as its ``__proto_dest__`` (:issue:`2533`, :pr:`2598`). A parse told to stop there -- at ``LinuxSLL``, @@ -41,7 +41,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and - :func:`~pcapkit.foundation.registry.protocols.register_frad` registers a + :func:`~pcapkit.foundation.registry.protocols.link.register_frad` registers a dissector for a further NLPID (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for @@ -55,7 +55,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` - unless :func:`~pcapkit.foundation.registry.protocols.register_netlink` + unless :func:`~pcapkit.foundation.registry.protocols.link.register_netlink` registers a dissector for its ``nlmsg_type`` (:pr:`2527`). * **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the @@ -74,21 +74,23 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- :class:`~pcapkit.protocols.link.radiotap.Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` - dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`). - It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) + dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`), + as do PCAP and PCAP-NG frames of ``LINKTYPE_IEEE802_11_RADIOTAP`` (127) + (:issue:`2608`, :pr:`2632`). It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, below, looked up in ``Radiotap``'s own registry under :attr:`LinkType.IEEE802_11 `, - and :func:`~pcapkit.foundation.registry.protocols.register_radiotap` + and :func:`~pcapkit.foundation.registry.protocols.link.register_radiotap` replaces that dissector (:issue:`2575`, :pr:`2607`). * **Added** -- :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which :class:`~pcapkit.protocols.link.radiotap.Radiotap` dispatches to - (:issue:`2575`, :pr:`2607`). It reads Frame Control, the Control Frame + (:issue:`2575`, :pr:`2607`), as do PCAP and PCAP-NG frames of + ``LINKTYPE_IEEE802_11`` (105) (:issue:`2608`, :pr:`2632`). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal @@ -108,7 +110,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain ``ICMP`` of :rfc:`792` as its ``id()`` and alias. - :func:`~pcapkit.foundation.registry.protocols.register_icmp_type` registers a + :func:`~pcapkit.foundation.registry.protocols.internet.register_icmp_type` registers a message-body parser. The type and code registries are :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. @@ -146,9 +148,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. gives (:rfc:`4340#section-9.2`): a coverage running past the packet is ``INVALID`` and warns, and a ``make()`` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. - :func:`~pcapkit.foundation.registry.protocols.register_dccp` registers a + :func:`~pcapkit.foundation.registry.protocols.transport.register_dccp` registers a port's dissector, and - :func:`~pcapkit.foundation.registry.protocols.register_dccp_option` an + :func:`~pcapkit.foundation.registry.protocols.transport.register_dccp_option` an option parser. The registries are :mod:`pcapkit.const.dccp`, crawled from IANA. @@ -168,7 +170,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``signature_algorithms``, ALPN, ``supported_versions`` and ``key_share``, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; - :func:`~pcapkit.foundation.registry.protocols.register_tls_extension` + :func:`~pcapkit.foundation.registry.protocols.application.register_tls_extension` registers a parser and constructor for a further type (:issue:`2514`, :pr:`2595`). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- @@ -181,8 +183,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also reads and builds DTLS 1.3's unified-header records (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays - Raw as an :rfc:`9146` ``tls12_cid`` record does (:issue:`2467`, - :pr:`2494`). Each whole DTLS record in a UDP datagram is likewise a layer of + Raw (:issue:`2467`, :pr:`2494`), and :rfc:`9146` ``tls12_cid`` records. A + :class:`~pcapkit.protocols.application.dtls.DTLSContext`, given to + ``extract()`` as its ``context``, resolves a ``tls12_cid`` record's + connection ID from those the capture's hellos negotiate in their + ``connection_id`` extension, which TLS and DTLS both decode, or from the + ``cid_length`` the caller gives it, and one it cannot resolve stays Raw + (:issue:`2515`, :pr:`2631`). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is @@ -234,6 +241,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`Datagram ` holds that decode as ``packet``, run on its first read as an IP or TCP datagram's is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). + Both run the classmethod + :meth:`DTLS.analyze_handshake ` + -- ``packet`` through the reassembly class's own ``__protocol_type__`` -- + which keeps a body that does not decode as raw octets, logging nothing + (:issue:`2622`, :pr:`2634`). * **Added** -- :func:`~pcapkit.foundation.analysis.ip.ip_datagram` and :func:`~pcapkit.foundation.analysis.tcp.tcp_datagram`, siblings of @@ -273,11 +285,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is ``NO_PSEUDO_HEADER``. -* **Added** -- :func:`~pcapkit.foundation.registry.protocols.register_ipx`, - :func:`~pcapkit.foundation.registry.protocols.register_loopback`, - :func:`~pcapkit.foundation.registry.protocols.register_sctp_chunk`, - :func:`~pcapkit.foundation.registry.protocols.register_sctp_parameter` and - :func:`~pcapkit.foundation.registry.protocols.register_sctp_cause`, wrapping +* **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, + :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, + :func:`~pcapkit.foundation.registry.protocols.transport.register_sctp_chunk`, + :func:`~pcapkit.foundation.registry.protocols.transport.register_sctp_parameter` and + :func:`~pcapkit.foundation.registry.protocols.transport.register_sctp_cause`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the @@ -322,6 +334,21 @@ HTTP/1 and HTTP/2 classes under the RFC names. A checksum given is written as given, and turning ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. +* **Changed** -- **a breaking change to** building a Quick-Start option in + :class:`~pcapkit.protocols.internet.ipv4.IPv4`, + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT`, + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` and + :class:`~pcapkit.protocols.transport.tcp.TCP`: a value its field cannot + carry raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x + built it (:issue:`2617`, :pr:`2630`). That is a negative or non-integer + rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an + IPv4 reserved value of 4 or more, and ``True`` as a TTL, nonce or + reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's + rounding of one down to zero (:issue:`1119`); a rate between two codes still + rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an + unassigned function code as the unassigned option their reader keeps it as, + as IPv4's does. + * **Changed** -- **a breaking change to** :class:`~pcapkit.protocols.application.drarp.DRARP` and :class:`~pcapkit.protocols.link.inarp.InARP`, which move from @@ -386,6 +413,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. place along, and :class:`~pcapkit.foundation.reassembly.ReassemblyManager` takes a required ``dtls`` (:issue:`2496`, :pr:`2528`). +* **Changed** -- :mod:`pcapkit.foundation.registry.protocols` is a package with + a submodule per layer -- :mod:`~pcapkit.foundation.registry.protocols.link`, + :mod:`~pcapkit.foundation.registry.protocols.internet`, + :mod:`~pcapkit.foundation.registry.protocols.transport`, + :mod:`~pcapkit.foundation.registry.protocols.application` and + :mod:`~pcapkit.foundation.registry.protocols.misc`, with the shared + registrars in :mod:`~pcapkit.foundation.registry.protocols.protocol` -- and + exports every public function they define, so each ``register_*`` imports + from it and from :mod:`pcapkit.foundation.registry` as before + (:issue:`2578`, :pr:`2635`). A registrar's ``__module__`` names its + submodule, and its log records carry that submodule's logger name, a child + of the package's, so configuring the package's logger still applies. + * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order (:issue:`2490`, :pr:`2491`, :pr:`2522`, :pr:`2523`, :pr:`2538`, @@ -426,7 +466,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``:issue:`` and ``:pr:`` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a - snapshot of the tracker (:issue:`2561`, :pr:`2626`). + snapshot of the tracker (:issue:`2561`, :pr:`2626`). A registry wrapper goes + in the submodule of :mod:`pcapkit.foundation.registry.protocols` for the + layer of the class owning its map (:issue:`2578`, :pr:`2635`). A protocol + class owns the parse of a reassembled outcome, which each + :mod:`pcapkit.foundation.analysis` function wraps without parsing of its + own, and a project test fails on a module under :mod:`pcapkit.protocols` + importing :mod:`pcapkit.foundation` outside ``TYPE_CHECKING``, + :mod:`pcapkit.foundation.registry.protocols` aside (:issue:`2622`, + :pr:`2634`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` From 8ab5026acb5c6084fb5df83aec1937d3d28f43fb Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 10:00:19 -0500 Subject: [PATCH 12/27] docs(changelog): fold NoPayload's move, family-base dispatch and the shared value helpers into 1.6.0 (#2449) - Breaking: NoPayload and its data model and schema move from misc.null to misc.no_payload with no re-export, so a frame pickled by 1.5.x fails to load (#2636). 1.5.0's three NoPayload roles and 1.6.0's own point at misc.no_payload, 1.5.0's text unchanged. - Added: the IP, L2TP, ICMP, IGMP and LinuxSLL bases give a member on read, make and from_data, version= naming it, with the refusals and IPv4's Version check ahead of the header (#2639). - Changed: SCTP's chunk-flag and Invalid Stream Identifier reserved-octet messages (#2638). Folded into the TLS/DTLS entry: make() raises StructError for a value too wide for its field (#2638), and the DTLS_CIDCiphertext and TLS_ConnectionId package exports (#2641). - The conventions entry records the version= dispatch rule (#2639) and ARP as the member-order reference (#2638). - No entry for #2638's private value helpers or the Ethernet and PCAP-NG PAT_MAC_ADDR, as for #1844's OSPF copy; #2640's docstring fixes; or the test-only #2620, #2637, #2645 and #2646. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 10 ++++- docs/source/changelog/1.5.0.rst | 18 +++++---- docs/source/changelog/1.6.0.rst | 65 ++++++++++++++++++++++++++++++--- 3 files changed, 78 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8641183f4d..0beefa253a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,9 +18,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. + - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)), and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves a `tls12_cid` record's connection ID from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)), and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves a `tls12_cid` record's connection ID from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw; `pcapkit.protocols.data` and `pcapkit.protocols.schema` export the record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. @@ -48,6 +50,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. +- **Changed** -- **a breaking change to** `NoPayload`, which moves from `pcapkit.protocols.misc.null` to `pcapkit.protocols.misc.no_payload`, and its data model and header schema from `pcapkit.protocols.data.misc.null` and `pcapkit.protocols.schema.misc.null` to `pcapkit.protocols.data.misc.no_payload` and `pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old paths, as a protocol class takes its module's name ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2636](https://github.com/JarryShaw/PyPCAPKit/pull/2636)). The class keeps its name, and `pcapkit.NoPayload` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails to load, as the pickle names the old module, though its pickled `info` loads. + - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. - **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. @@ -58,9 +62,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- `pcapkit.foundation.registry.protocols` is a package with a submodule per layer -- `link`, `internet`, `transport`, `application` and `misc`, with the shared registrars in `protocol` -- and exports every public function they define, so each `register_*` imports from it and from `pcapkit.foundation.registry` as before ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A registrar's `__module__` names its submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. +- **Changed** -- `SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as `SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''` and `SCTP: invalid [Cause 1] reserved: must be 2 octets, not b''`, where they read `SCTP: [Chunk 132] invalid flags: must be one octet, not b''` and `SCTP: [Cause 1] invalid reserved: must be two octets, not b''` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A one-character `str` given as flags, which passed the check and failed to pack, is refused the same way. + - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index c8b7d45ea0..0f26476a47 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -1510,12 +1510,13 @@ Changed trailer bytes are not, so ``_decode_next_layer`` and ``_import_next_layer`` accept the ``-1`` sentinel -- the rest of the packet, resolving to :class:`~pcapkit.protocols.misc.raw.Raw`, or to - :class:`~pcapkit.protocols.misc.null.NoPayload` when nothing remains -- and - ``__post_init__`` keeps a payload that ``read`` set instead of overwriting it with - :class:`~pcapkit.protocols.misc.null.NoPayload`. A real protocol number (any ``proto`` - other than ``-1``) is still refused with :exc:`.UnsupportedCall`. A subclass that - calls either method with ``-1`` changes from raising to succeeding; no subclass in the - package does (:issue:`719`). + :class:`NoPayload ` + when nothing remains -- and ``__post_init__`` keeps a payload that ``read`` set + instead of overwriting it with + :class:`NoPayload `. + A real protocol number (any ``proto`` other than ``-1``) is still refused with + :exc:`.UnsupportedCall`. A subclass that calls either method with ``-1`` changes + from raising to succeeding; no subclass in the package does (:issue:`719`). * the :class:`~pcapkit.protocols.protocol.Protocol` and :class:`.ProtocolMeta` docstrings. :class:`~pcapkit.protocols.protocol.Protocol` claimed to be the base of every protocol family, though nothing in the package subclasses it, and @@ -2610,8 +2611,9 @@ Fixed the fabricated octets. :meth:`.IPv4.read` handed UDP the declared payload length when nothing past the IPv4 header was captured, so ``test.pcapng`` frame 3 rebuilt with 8 invented UDP octets (:issue:`1167`). Every ``_import_next_layer`` now goes through - one guard: no captured octets gives :class:`~pcapkit.protocols.misc.null.NoPayload`, - and fewer than the next layer's ``__length_hint__`` gives + one guard: no captured octets gives + :class:`NoPayload `, and fewer + than the next layer's ``__length_hint__`` gives :class:`~pcapkit.protocols.misc.raw.Raw` unless the parsed header fits anyway, as a 12-octet L2TPv2 ZLB does. A header-only IPv6 packet is ``IPv6`` rather than ``IPv6:UDP:Raw``, and frame 3 is ``Ethernet:Raw``, rebuilt byte for byte diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index aa4bffa393..d3d050503b 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -137,6 +137,26 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +* **Added** -- construction through a versioned family's base: + :class:`~pcapkit.protocols.internet.ip.IP` and + :class:`~pcapkit.protocols.link.l2tp.L2TP`, which raised :exc:`TypeError` in + 1.5.x, and the :class:`~pcapkit.protocols.internet.icmp.ICMP`, + :class:`~pcapkit.protocols.internet.igmp.IGMP` and + :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` bases above give a member, + never an instance of the base, on parsing octets, building from fields and + ``from_data`` (:issue:`2574`, :pr:`2639`). The ``version`` keyword names the + member. Without it, IP's, IGMP's and L2TP's octets give the member they + describe, and ``from_data`` the one its data names, IGMP's data models + carrying a ``version`` for it. Building from fields without it raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError`, whose message asks for + ``version``, and so do ICMP and Linux cooked-capture octets, which carry no + version; their ``from_data`` on a :class:`dict` raises it too, as does a + version a family has no member for, such as L2TP's ``3``. A member built + directly takes ``version`` as well, which must be its own. IPv4 and IPv6 + check the Version field before the rest of the header, so octets of the other + version raise ``[IPv4] invalid version: 6`` say, where IPv4's error named + whichever field they broke first. + * **Added** -- :class:`~pcapkit.protocols.transport.dccp.DCCP` (:rfc:`4340`) for IP protocol 33, which was kept as :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2489`, :pr:`2593`). It @@ -172,7 +192,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. body its parser rejects, which logs nothing; :func:`~pcapkit.foundation.registry.protocols.application.register_tls_extension` registers a parser and constructor for a further type (:issue:`2514`, - :pr:`2595`). Each whole TLS record in a TCP segment is a layer of its own, + :pr:`2595`). ``make()`` refuses a value too wide for its field, in the record + header or in the content it builds, with + :exc:`~pcapkit.utilities.exceptions.StructError` (:issue:`2614`, :pr:`2638`). + Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment @@ -188,8 +211,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``extract()`` as its ``context``, resolves a ``tls12_cid`` record's connection ID from those the capture's hellos negotiate in their ``connection_id`` extension, which TLS and DTLS both decode, or from the - ``cid_length`` the caller gives it, and one it cannot resolve stays Raw - (:issue:`2515`, :pr:`2631`). Each whole DTLS record in a UDP datagram is likewise a layer of + ``cid_length`` the caller gives it, and one it cannot resolve stays Raw; + :mod:`pcapkit.protocols.data` and :mod:`pcapkit.protocols.schema` export the + record's classes as ``DTLS_CIDCiphertext`` and the extension's data model as + ``TLS_ConnectionId`` (:issue:`2515`, :pr:`2631`, :pr:`2641`). Each whole DTLS + record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is @@ -360,6 +386,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``pcapkit.InARP``, their :mod:`pcapkit.protocols` exports and a parsed RARP's ``DRARP`` or ARP's ``InARP`` alias are unchanged. +* **Changed** -- **a breaking change to** + :class:`~pcapkit.protocols.misc.no_payload.NoPayload`, which moves from + ``pcapkit.protocols.misc.null`` to :mod:`pcapkit.protocols.misc.no_payload`, + and its data model and header schema from ``pcapkit.protocols.data.misc.null`` + and ``pcapkit.protocols.schema.misc.null`` to + :mod:`pcapkit.protocols.data.misc.no_payload` and + :mod:`pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old + paths, as a protocol class takes its module's name (:issue:`2529`, + :pr:`2636`). The class keeps its name, and ``pcapkit.NoPayload`` and the + package exports of that name are unchanged. A frame pickled by 1.5.x fails + to load, as the pickle names the old module, though its pickled ``info`` + loads. + * **Changed** -- **a breaking change to** HTTP: parsing octets through :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields and its ``from_data`` give the @@ -426,6 +465,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. +* **Changed** -- :class:`~pcapkit.protocols.transport.sctp.SCTP`'s chunk makers + refuse flags that are not one octet, and its Invalid Stream Identifier cause + maker reserved octets that are not two, as + ``SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''`` and + ``SCTP: invalid [Cause 1] reserved: must be 2 octets, not b''``, where they + read ``SCTP: [Chunk 132] invalid flags: must be one octet, not b''`` and + ``SCTP: [Cause 1] invalid reserved: must be two octets, not b''`` + (:issue:`2614`, :pr:`2638`). A one-character :class:`str` given as flags, which + passed the check and failed to pack, is refused the same way. + * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order (:issue:`2490`, :pr:`2491`, :pr:`2522`, :pr:`2523`, :pr:`2538`, @@ -474,7 +523,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. own, and a project test fails on a module under :mod:`pcapkit.protocols` importing :mod:`pcapkit.foundation` outside ``TYPE_CHECKING``, :mod:`pcapkit.foundation.registry.protocols` aside (:issue:`2622`, - :pr:`2634`). + :pr:`2634`). A versioned family's base takes its member from the ``version`` + keyword, or else from the octets or the data model, never from a field, has no + default member, and refuses octets that carry no version unless ``version`` + names one (:issue:`2574`, :pr:`2639`). The member-order convention's reference + class is :class:`~pcapkit.protocols.link.arp.ARP`, where it was + :class:`~pcapkit.protocols.link.ethernet.Ethernet`, whose MAC address helpers + moved to the base class (:issue:`2614`, :pr:`2638`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -547,7 +602,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- a membership test such as ``'ESP' in layer`` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised :exc:`AttributeError`. - The layer now holds :class:`~pcapkit.protocols.misc.null.NoPayload`, the + The layer now holds :class:`~pcapkit.protocols.misc.no_payload.NoPayload`, the upper layer belonging to the enclosing IPv6 header (:issue:`2484`, :issue:`2487`, :pr:`2486`, :pr:`2488`). From 777f7088ceb7bc2074e8a40626b525f26a587719 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 10:25:57 -0500 Subject: [PATCH 13/27] docs(changelog): fold DTLS 1.3 connection IDs and the 1.5.0 HTTP roles into 1.6.0 (#2449) - Folded into the TLS/DTLS entry: DTLSContext resolves a unified-header record's connection ID as a tls12_cid one's, an ID NewConnectionId issues coming only from the caller; every unified-header record's info carries cid, None when C is clear, the one-data-model ruling on #2625; make() takes cid (#2643). The entry drops the stale "stays Raw" for those records. - 1.5.0's httpv1.HTTP and httpv2.HTTP roles point at HTTPv1 and HTTPv2, renamed by #2544, so the nitpicky build no longer warns on them; 1.5.0's text unchanged. - No entry for the test-only #2644. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 2 +- docs/source/changelog/1.5.0.rst | 14 +++++++------- docs/source/changelog/1.6.0.rst | 27 +++++++++++++++++---------- 3 files changed, 25 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0beefa253a..7c75d60f5c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,7 +22,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds DTLS 1.3's unified-header records ([RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1)), except one carrying a connection ID, which stays Raw ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)), and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves a `tls12_cid` record's connection ID from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw; `pcapkit.protocols.data` and `pcapkit.protocols.schema` export the record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. `pcapkit.protocols.data` and `pcapkit.protocols.schema` export a `tls12_cid` record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index 0f26476a47..ac2196687c 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -2272,7 +2272,7 @@ Fixed ``code=129``) was deleted once that registry defect was fixed as :issue:`690`. ``HIP_COPIES`` dropped to one as :issue:`689`, once :issue:`672` and :issue:`679` left it routing around nothing. -* :class:`httpv1.HTTP ` raised a bare +* :class:`httpv1.HTTP ` raised a bare exception for any payload that is not an HTTP/1 message, at these sites: the header/body split in ``read``, the header's CRLF split and the start-line split in ``_read_http_header``, and ``item[1]`` on a field line with no colon. @@ -2280,13 +2280,13 @@ Fixed HTTP/1 attempt aborted the guess and the HTTP/2 arm below was dead code for each of those inputs. Each site now raises :exc:`.ProtocolError`, which each method's ``Raises:`` section documented, and a test pins the invariant (nothing escapes - :class:`httpv1.HTTP ` that is not a + :class:`httpv1.HTTP ` that is not a :exc:`.ProtocolError`), so any further bare-raising site would fail there. An obs-fold continuation line [:rfc:`9112#section-5.2`] is legal HTTP/1 and is now unfolded, the RFC's remedy, rather than read as a field of its own (with no colon it raised that ``IndexError``; with one it parsed into a spurious extra field). Reaching the HTTP/2 arm exposed a payload under nine octets to - :class:`httpv2.HTTP `, where it usually + :class:`httpv2.HTTP `, where it usually fails inside the schema with a bare ``struct.error`` (not a ``ValueError``, :exc:`.ProtocolError` or pcapkit :exc:`.StructError`, so uncatchable); ``read`` converts it and ``_guess_version`` suppresses it on the last arm only, since an arm @@ -2311,8 +2311,8 @@ Fixed ``examples/generators/pcapng.py``'s ``_journal_entry`` cited a bare line number for the defect its alignment workaround avoids and mis-attributed the fix; it now names :issue:`678`'s ``not line.strip(b'\x00')`` guard and the mechanism (:issue:`791`). -* :meth:`.httpv2.HTTP.read` tested only the 24-bit declared length off the wire - (``schema.length < 9``), never how many octets the buffer held, so a frame backed by +* :meth:`httpv2.HTTP.read() ` + tested only the 24-bit declared length off the wire (``schema.length < 9``), never how many octets the buffer held, so a frame backed by far fewer real octets than declared parsed and reported the declared, attacker-controlled length. Sweeping the declared length against a four-octet buffer gave non-monotone accept/reject (9, 15, 65535 and 16777215 parsed, 10 and 100 did @@ -2375,7 +2375,7 @@ Fixed safe heuristic, since a ``type <= 9`` guess misfires on binary HTTP/1 bodies). Protochain over the sample captures is unchanged (:issue:`800`). * :attr:`.TCP.__proto__` bound - :class:`httpv1.HTTP ` directly for ports 80 + :class:`httpv1.HTTP ` directly for ports 80 and 8080, so a segment on either port was HTTP/1 by assertion of the port alone; both versions share those ports, so the payload has to decide. Repointed to the generic :class:`pcapkit.protocols.application.http.HTTP` proxy, whose identification became @@ -2474,7 +2474,7 @@ Fixed ``[if_rxspeed]`` rather than ``[if_txspeed]``, and ``_read_option_epb_queue``'s length guard says ``[epb_queue]`` rather than ``[epb_packetid]``. Only message text changes; the exception and warning types are unchanged (:issue:`1038`). -* :class:`httpv1.HTTP ` refused every HTTP/1 +* :class:`httpv1.HTTP ` refused every HTTP/1 message with no header fields, which [:rfc:`9112#section-2.1`] allows (``*( field-line CRLF )``): ``b'GET / HTTP/1.0\r\n\r\n'`` raised ``ProtocolError: HTTP: invalid format``, because ``_read_http_header`` treated a diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index d3d050503b..0e2292acea 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -204,17 +204,24 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.protocols.application.sslv3` (:issue:`2529`, :pr:`2602`), exists for explicit construction only. A record split across TCP segments is not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also - reads and builds DTLS 1.3's unified-header records - (:rfc:`9147#section-4.1`), except one carrying a connection ID, which stays - Raw (:issue:`2467`, :pr:`2494`), and :rfc:`9146` ``tls12_cid`` records. A + reads and builds the DTLS 1.3 unified-header records of + :rfc:`9147#section-4.1` (:issue:`2467`, :pr:`2494`) and :rfc:`9146` + ``tls12_cid`` records. A :class:`~pcapkit.protocols.application.dtls.DTLSContext`, given to - ``extract()`` as its ``context``, resolves a ``tls12_cid`` record's - connection ID from those the capture's hellos negotiate in their - ``connection_id`` extension, which TLS and DTLS both decode, or from the - ``cid_length`` the caller gives it, and one it cannot resolve stays Raw; - :mod:`pcapkit.protocols.data` and :mod:`pcapkit.protocols.schema` export the - record's classes as ``DTLS_CIDCiphertext`` and the extension's data model as - ``TLS_ConnectionId`` (:issue:`2515`, :pr:`2631`, :pr:`2641`). Each whole DTLS + ``extract()`` as its ``context``, resolves the connection ID of a + ``tls12_cid`` record, or of a unified-header record with its ``C`` bit set + (:issue:`2625`, :pr:`2643`), from those the capture's hellos negotiate in + their ``connection_id`` extension, which TLS and DTLS both decode, or from + the ``cid_length`` the caller gives it, and one it cannot resolve stays Raw. + An ID a DTLS 1.3 ``NewConnectionId`` issues is encrypted, so only the caller + can give it. A unified-header record's ``info`` carries its ID as ``cid``, + :data:`None` when the ``C`` bit is clear, as ``length`` is when the ``L`` bit + is, so that the one header keeps one data model, as settled on + :issue:`2625`; its ``make()`` takes a ``cid`` and sets the ``C`` bit. + :mod:`pcapkit.protocols.data` and :mod:`pcapkit.protocols.schema` export a + ``tls12_cid`` record's classes as ``DTLS_CIDCiphertext`` and the extension's + data model as ``TLS_ConnectionId`` (:issue:`2515`, :pr:`2631`, :pr:`2641`). + Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to From c90a1251e9543aad3b6f31f4d5454ccb7bff9901 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 11:24:57 -0500 Subject: [PATCH 14/27] docs(changelog): fold HTTP's versioned-base rules, the IGMP and GRE checksums and PyShark's lost tshark status into 1.6.0 (#2449) - Breaking, in the HTTP entry: building HTTP from fields takes version= and from_data refuses a dict, where 1.5.x built HTTP/1.1 and took one; HTTPv1 refuses preface, which 1.5.x wrote ahead of the message (#2651). The versioned-base entry and the conventions entry point at it. - Folded into the checksum entry: IGMP and GRE verify and compute their checksums, verify_message, GRE's checksum_present, and a cut-short GRE frame reading INVALID (#2654). The contextvars entry drops in_fragment, private to the checksum module and in no release. - Fixed: the PyShark engine absorbs the TSharkCrashException asyncio's lost tshark exit status raises, on every Python (#2642). - No entry for the test-only #2592. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 12 ++++---- docs/source/changelog/1.6.0.rst | 49 +++++++++++++++++++++++++-------- 2 files changed, 45 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c75d60f5c..1188569095 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,7 +18,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. -- **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. +- **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. HTTP's base follows the same rules, a breaking change given below ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. @@ -34,13 +34,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so a frame the capture cut short reads as `INVALID`. - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). - **Added** -- `pcapkit.corekit.fields.strings.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)). -- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `in_fragment` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). +- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. @@ -52,7 +52,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** `NoPayload`, which moves from `pcapkit.protocols.misc.null` to `pcapkit.protocols.misc.no_payload`, and its data model and header schema from `pcapkit.protocols.data.misc.null` and `pcapkit.protocols.schema.misc.null` to `pcapkit.protocols.data.misc.no_payload` and `pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old paths, as a protocol class takes its module's name ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2636](https://github.com/JarryShaw/PyPCAPKit/pull/2636)). The class keeps its name, and `pcapkit.NoPayload` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails to load, as the pickle names the old module, though its pickled `info` loads. -- **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`. +- **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). Building it from fields takes `version`, `1` or `2`, as the other versioned bases above do, and raises `ProtocolError` without it, where 1.5.x built an HTTP/1.1 message; its `from_data` takes the member from the data model's class and refuses a `dict`, where 1.5.x took one ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`, which `HTTPv1` refuses, where 1.5.x wrote the HTTP/2 preface ahead of the HTTP/1 message ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). - **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. @@ -66,7 +66,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). @@ -84,6 +84,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- the PyPCAPFile engine's header checks reported the warnings of a trial parse they rejected. The extension-header trial now drops them as under `beholder`, and the IPv4 and TCP header check holds its trial's and discards them when it raises ([#2526](https://github.com/JarryShaw/PyPCAPKit/issues/2526), [#2577](https://github.com/JarryShaw/PyPCAPKit/pull/2577)). +- **Fixed** -- an extraction with the PyShark engine could end in pyshark's `TSharkCrashException` with `retcode: 255` for a `tshark` that exited cleanly, when a `Popen.poll()` reaped it before the `asyncio` child watcher could, which then reported `255` (CPython gh-87744). Closing the capture absorbs that crash, logging it at `logging.DEBUG`, on every Python, when the capture was read to its end and each `tshark` it lists reports `255` to `asyncio` and `0` to its `Popen`; any other error is raised ([#2633](https://github.com/JarryShaw/PyPCAPKit/issues/2633), [#2642](https://github.com/JarryShaw/PyPCAPKit/pull/2642)). + - **Fixed** -- selecting `'IPv6_Ext'` matched none of the dedicated extension headers: `IPv6_Ext.id()` now lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH and ESP do under IPsec ([#2462](https://github.com/JarryShaw/PyPCAPKit/issues/2462), [#2464](https://github.com/JarryShaw/PyPCAPKit/pull/2464)). - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 0e2292acea..5cb69ef701 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -152,7 +152,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``version``, and so do ICMP and Linux cooked-capture octets, which carry no version; their ``from_data`` on a :class:`dict` raises it too, as does a version a family has no member for, such as L2TP's ``3``. A member built - directly takes ``version`` as well, which must be its own. IPv4 and IPv6 + directly takes ``version`` as well, which must be its own. HTTP's base follows + the same rules, a breaking change given below (:issue:`2628`, :pr:`2651`). + IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise ``[IPv4] invalid version: 6`` say, where IPv4's error named whichever field they broke first. @@ -301,7 +303,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- verification and computation of the one's-complement Internet checksum (:rfc:`1071`) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in :mod:`pcapkit.corekit.checksum.internet` (:issue:`2470`, :issue:`2472`, - :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`), where + :issue:`2473`, :issue:`2474`, :issue:`2475`, :pr:`2543`), and of IGMP and + GRE (:issue:`2618`, :pr:`2654`), where :mod:`pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from :mod:`pcapkit.protocols` (:issue:`2580`, :pr:`2586`). Parsing verifies only under ``PCAPKIT_CHECKSUM_VERIFY``, off by default, @@ -315,8 +318,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` it builds, and ``make()`` of TCP, UDP and ICMPv6 also takes a ``pseudo``. What checksum offload leaves on the sending host is ``NOT_COMPUTED`` rather than a - mismatch; a fragment's TCP, UDP, ICMP or ICMPv6 checksum, which covers - octets the fragment lacks, is ``NO_PSEUDO_HEADER``. + mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which + covers octets the fragment lacks, is ``NO_PSEUDO_HEADER``. + :func:`~pcapkit.corekit.checksum.internet.verify_message` verifies a + checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, + giving ``NO_PSEUDO_HEADER`` for a fragment's payload. GRE verifies only with + its ``C`` bit set, and is ``NOT_COMPUTED`` with it clear; its ``make()`` + takes ``checksum_present`` to set the bit and compute the checksum. GRE + states no length, so a frame the capture cut short reads as ``INVALID``. * **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, @@ -340,7 +349,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- :mod:`pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- ``beholding``, - ``frame_depth``, ``in_fragment`` and ``rebuilding`` -- and importing nothing + ``frame_depth`` and ``rebuilding`` -- and importing nothing from :mod:`pcapkit` (:issue:`2616`, :pr:`2623`). * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a @@ -413,7 +422,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`~pcapkit.protocols.application.httpv2.HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under ``httpv1`` or ``httpv2`` where it sat under ``http`` - (:issue:`2524`, :pr:`2544`). The members are renamed after their modules, + (:issue:`2524`, :pr:`2544`). Building it from fields takes ``version``, + ``1`` or ``2``, as the other versioned bases above do, and raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError` without it, where 1.5.x + built an HTTP/1.1 message; its ``from_data`` takes the member from the data + model's class and refuses a :class:`dict`, where 1.5.x took one + (:issue:`2628`, :pr:`2651`). The members are renamed after their modules, ``httpv1.HTTP`` and ``httpv2.HTTP`` becoming ``HTTPv1`` and ``HTTPv2`` along with their schema and data classes, with no re-export at the old paths (:issue:`2529`); ``pcapkit.HTTPv1``, ``pcapkit.HTTPv2`` and the package @@ -432,7 +446,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. passes for an HTTP/2 frame header but ``HTTPv2`` refuses carries that parser's own error on either path, where it was ``'HTTP: unknown version'``. ``HTTPv2`` itself now parses a payload opening with the connection preface, - kept as ``preface``, and its ``make()`` takes ``preface``. + kept as ``preface``, and its ``make()`` takes ``preface``, which ``HTTPv1`` + refuses, where 1.5.x wrote the HTTP/2 preface ahead of the HTTP/1 message + (:issue:`2628`, :pr:`2651`). * **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons @@ -533,10 +549,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. :pr:`2634`). A versioned family's base takes its member from the ``version`` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless ``version`` - names one (:issue:`2574`, :pr:`2639`). The member-order convention's reference - class is :class:`~pcapkit.protocols.link.arp.ARP`, where it was - :class:`~pcapkit.protocols.link.ethernet.Ethernet`, whose MAC address helpers - moved to the base class (:issue:`2614`, :pr:`2638`). + names one (:issue:`2574`, :pr:`2639`); HTTP's base follows that rule, its + ``from_data`` reading the data model's class, since the HTTP data models + carry no ``version`` (:issue:`2628`, :pr:`2651`). The member-order + convention's reference class is :class:`~pcapkit.protocols.link.arp.ARP`, + where it was :class:`~pcapkit.protocols.link.ethernet.Ethernet`, whose MAC + address helpers moved to the base class (:issue:`2614`, :pr:`2638`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -599,6 +617,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. under ``beholder``, and the IPv4 and TCP header check holds its trial's and discards them when it raises (:issue:`2526`, :pr:`2577`). +* **Fixed** -- an extraction with the PyShark engine could end in pyshark's + ``TSharkCrashException`` with ``retcode: 255`` for a ``tshark`` that exited + cleanly, when a ``Popen.poll()`` reaped it before the :mod:`asyncio` child + watcher could, which then reported ``255`` (CPython gh-87744). Closing the + capture absorbs that crash, logging it at :data:`logging.DEBUG`, on every + Python, when the capture was read to its end and each ``tshark`` it lists + reports ``255`` to :mod:`asyncio` and ``0`` to its ``Popen``; any other + error is raised (:issue:`2633`, :pr:`2642`). + * **Fixed** -- selecting ``'IPv6_Ext'`` matched none of the dedicated extension headers: :meth:`IPv6_Ext.id() ` now From cb68629bf1575c0008f89b1b3fabc3173c9fd256 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 13:44:01 -0500 Subject: [PATCH 15/27] docs(changelog): fold the range-checked makers, the registry index, the cut-frame fixes and GRE's bindings into 1.6.0 (#2449) - Breaking: TCP, UDP and SCTP makers (#2670) and PCAP-NG's (#2671) raise on a value a field cannot hold, where 1.5.x wrote it masked; each entry gives the old and new values. The TTL Diff wraps as before. - Changed: __index__ returns the LinkType, EtherType, TransType or AppType value, ports as ints (#2652); only Create Release waits for a deployment approval (#2658). The conventions entry records the index rule. - Fixed: a cut HTTP/2 SETTINGS frame parses and rebuilds; HTTPv2.make takes frame_length (#2666). - Folded into unreleased entries: FRAD at LinkType 107 and GRE at IP protocol 47 (#2667); cut IGMPv3 messages and num_group_records (#2662, #2672); Linux_ARPHRD (#2663); DCCP's range checks (#2670); a cut GRE frame is TRUNCATED, replacing fold-12's INVALID note, and frame_length in pcapkit.utilities.contextvars (#2668). - No entry for _make_sint's private allow_unsigned, ARP's private PAT_MAC_ADDR, or the test- and docs-only d728873ca, 326a6cfd8, fc80fb1cb, a37edf1e3 and 27667075a. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 26 +++++--- docs/source/changelog/1.6.0.rst | 102 +++++++++++++++++++++++++++++--- 2 files changed, 111 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1188569095..09bcde4cf1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,13 +2,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the protocol pairs that wanted a shared abstract base, and the two that simply had no dissector at all. Alongside it come the Linux cooked-capture headers and a dissector for each `arphrd_type` they do not read as an EtherType, the IP pseudo-header, verified and computed one's-complement checksums, stricter handling of a malformed IPv6 extension header, and HTTP dispatched to its HTTP/1 and HTTP/2 classes under the RFC names. -- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)), as does a protocol class declaring `code=ARPHRD.X`, since `ARPHRD` names `LinuxSLL` as its `__proto_dest__` ([#2533](https://github.com/JarryShaw/PyPCAPKit/issues/2533), [#2598](https://github.com/JarryShaw/PyPCAPKit/pull/2598)). A parse told to stop there -- at `LinuxSLL`, the version itself or the link layer -- keeps the payload `Raw` on either path ([#2532](https://github.com/JarryShaw/PyPCAPKit/issues/2532), [#2587](https://github.com/JarryShaw/PyPCAPKit/pull/2587)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)). +- **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)), as does a protocol class declaring `code=ARPHRD.X`, since `ARPHRD` names `LinuxSLL` as its `__proto_dest__` ([#2533](https://github.com/JarryShaw/PyPCAPKit/issues/2533), [#2598](https://github.com/JarryShaw/PyPCAPKit/pull/2598)). A parse told to stop there -- at `LinuxSLL`, the version itself or the link layer -- keeps the payload `Raw` on either path ([#2532](https://github.com/JarryShaw/PyPCAPKit/issues/2532), [#2587](https://github.com/JarryShaw/PyPCAPKit/pull/2587)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)), and `pcapkit.const` exports it as `Linux_ARPHRD` ([#2661](https://github.com/JarryShaw/PyPCAPKit/issues/2661), [#2663](https://github.com/JarryShaw/PyPCAPKit/pull/2663)). -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)), as do PCAP and PCAP-NG frames of `LINKTYPE_FRELAY` (107) ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). - **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as `Raw` unless `register_netlink()` registers a dissector for its `nlmsg_type` ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). -- **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)). The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. +- **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)), as does an IP layer for IP protocol 47, through the registry (`Internet.__proto__`) IPv4 and IPv6 share ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). So with reassembly and tracing on, analysis follows a GRE tunnel: TCP inside GRE over IP reassembles under its inner addresses, and a fragmented GRE datagram reassembles to a parsed GRE layer. The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. - **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). @@ -16,11 +16,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses as far as its octets go and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. - **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. HTTP's base follows the same rules, a breaking change given below ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. -- **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. +- **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. Its `make()` refuses a value a field cannot hold with `ProtocolError` naming the field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). - **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. `pcapkit.protocols.data` and `pcapkit.protocols.schema` export a `tls12_cid` record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). @@ -34,13 +34,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so a frame the capture cut short reads as `INVALID`. +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). - **Added** -- `pcapkit.corekit.fields.strings.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)). -- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). +- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `frame_length` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). `frame_length` holds the captured and original length of the frame being dissected, as its PCAP record or PCAP-NG packet block states them, and is `None` outside a frame and inside an IP layer's payload ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. @@ -48,6 +48,10 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** building a Quick-Start option in `IPv4`, `HOPOPT`, `IPv6_Opts` and `TCP`: a value its field cannot carry raises `ProtocolError`, where 1.5.x built it ([#2617](https://github.com/JarryShaw/PyPCAPKit/issues/2617), [#2630](https://github.com/JarryShaw/PyPCAPKit/pull/2630)). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and `True` as a TTL, nonce or reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. +- **Changed** -- **a breaking change to** building `TCP`, `UDP` and `SCTP`: a header or option field given an integer it cannot hold raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and a fixed-size octet field given the wrong number of octets raises, where it was padded or cut ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). So `TCP(window=70000)` raises `TCP: invalid window: 70000`, where it wrote `0x1170`; `TCP(seq_no=-1)` and `SCTP(vtag=-1)` raise, where they wrote `0xFFFFFFFF`; a UDP payload of `0xFFF8` octets raises, where its Length was written `0`; and `TCP(checksum=b'\x01')` raises, where it wrote `0100`. The TCP User Timeout refuses a negative, `True` or non-numeric timeout by name, where a negative one raised a bare `ValueError` and `True` built one second. The Quick-Start TTL Diff, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) defines modulo 256, wraps as in 1.5.x. + +- **Changed** -- **a breaking change to** building `PCAPNG` blocks: a field given a value it cannot hold raises `ProtocolError`, where 1.5.x wrote it masked ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2671](https://github.com/JarryShaw/PyPCAPKit/pull/2671)). The Section Header Block's Section Length takes `-2**63` to `2**64 - 1`, `-1` and `0xFFFFFFFFFFFFFFFF` writing the same octets, and refuses anything else, where `2**64` wrote `0`, `True` wrote `1` and `None` the `-1` sentinel. The obsolete Packet Block's Drops Count takes `0` to `0xFFFF`, where `-1` wrote `0xFFFF`, `0x10000` wrote `0` and `True` wrote `1`. The `if_tzone` and `if_tsoffset` options refuse a value outside their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads `PCAP-NG: invalid [IDB] snaplen: 4294967296`, where it read `PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296`. + - **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. - **Changed** -- **a breaking change to** `NoPayload`, which moves from `pcapkit.protocols.misc.null` to `pcapkit.protocols.misc.no_payload`, and its data model and header schema from `pcapkit.protocols.data.misc.null` and `pcapkit.protocols.schema.misc.null` to `pcapkit.protocols.data.misc.no_payload` and `pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old paths, as a protocol class takes its module's name ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2636](https://github.com/JarryShaw/PyPCAPKit/pull/2636)). The class keeps its name, and `pcapkit.NoPayload` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails to load, as the pickle names the old module, though its pickled `info` loads. @@ -64,9 +68,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- `SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as `SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''` and `SCTP: invalid [Cause 1] reserved: must be 2 octets, not b''`, where they read `SCTP: [Chunk 132] invalid flags: must be one octet, not b''` and `SCTP: [Cause 1] invalid reserved: must be two octets, not b''` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A one-character `str` given as flags, which passed the check and failed to pack, is refused the same way. +- **Changed** -- a protocol's `__index__` returns its value in the LinkType, EtherType, TransType or AppType registry wherever one names it, and a class an AppType member names returns that member's port as an `int` ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). So `L2TPv2`, `FTP`, `FTP_DATA`, the HTTP classes and `NGAP`, which raised in 1.5.x, return 1701, 21, 20, 80 and 38412, and TLS, SSLv3 and DTLS the ports TCP and UDP dispatch them at, 443 and 5684; Radiotap, FRAD and GRE return their LinkType or TransType member. SSLv2 and the SSL base, which nothing dispatches to, raise as before, and no value 1.5.x returned changes. + - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). @@ -76,6 +82,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- a weekly Issue and PR Kinds workflow regenerates, with `util/issue_pr_kinds.py`, the snapshot of the tracker that the project test of each role's kind reads, and opens or updates a pull request from `bot/issue-pr-kinds` when it changes, never pushing `main` ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). +- **Changed** -- of the workflows, only Create Release waits for a deployment approval: the `pr` jobs of the Issue and PR Kinds and Python Versions workflows read the release App's key from the `release-app` environment, which has no required reviewer, where they entered `github-release` and waited on its approval, and skip with a warning while that environment holds no key ([#2648](https://github.com/JarryShaw/PyPCAPKit/issues/2648), [#2658](https://github.com/JarryShaw/PyPCAPKit/pull/2658)). + - **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and `CONTRIBUTING.md` and the pull request template say so ([#2449](https://github.com/JarryShaw/PyPCAPKit/issues/2449), [#2483](https://github.com/JarryShaw/PyPCAPKit/pull/2483)). - **Fixed** -- `AppType.get` takes the `default` `EnumLookup.get` advertises, a registered member's value that it never mints, and a miss raises `EnumValueError`, still a `ValueError`, where it raised a builtin one ([#2545](https://github.com/JarryShaw/PyPCAPKit/issues/2545), [#2568](https://github.com/JarryShaw/PyPCAPKit/pull/2568)). @@ -90,6 +98,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). +- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses as far as its octets go and rebuilds as captured, and `HTTPv2`'s `make()` takes `frame_length`, from the frame's own length to `0xFFFFFF`, so that the declared Length survives ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666)). + - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). --- diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 5cb69ef701..3341d6abe6 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -32,12 +32,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the ```` values that table lists, so any other, ``ARPHRD_ETHER`` included, reads as ``Unassigned`` (:issue:`2495`, - :pr:`2497`). + :pr:`2497`), and :mod:`pcapkit.const` exports it as ``Linux_ARPHRD`` + (:issue:`2661`, :pr:`2663`). * **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`), which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for - ``ARPHRD_FRAD`` (:pr:`2499`). The direct NLPID and SNAP-encapsulated forms + ``ARPHRD_FRAD`` (:pr:`2499`), as do PCAP and PCAP-NG frames of + ``LINKTYPE_FRELAY`` (107) (:issue:`2649`, :pr:`2667`). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and @@ -64,7 +66,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for both ``ARPHRD_IPGRE`` and ``ARPHRD_IP6GRE`` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version (:issue:`2501`, - :pr:`2508`). The Protocol Type field dispatches onward as an EtherType, + :pr:`2508`), as does an IP layer for IP protocol 47, through the registry + (:attr:`Internet.__proto__ `) + IPv4 and IPv6 share (:issue:`2649`, :pr:`2667`). So with reassembly and + tracing on, analysis follows a GRE tunnel: TCP inside GRE over IP + reassembles under its inner addresses, and a fragmented GRE datagram + reassembles to a parsed GRE layer. The Protocol Type field dispatches onward as an EtherType, through the family registry (:attr:`Link.__proto__ `) Ethernet, VLAN and LinuxSLL share. A non-zero Version, :rfc:`2637`'s @@ -125,7 +132,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. otherwise, one of 12 or more IGMPv3 (:issue:`2439`, :issue:`2448`, :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and - QQIC are decoded from their floating-point form. Any other message, a 9- to + QQIC are decoded from their floating-point form. An IGMPv3 message the + capture cut short, a Report cut inside its group records included, parses as + far as its octets go and rebuilds as captured, whether parsed on its own or + carried by IPv4, and IGMPv3's ``make()`` takes ``num_group_records``, from + the record count to ``0xFFFF`` (:issue:`2650`, :issue:`2660`, :pr:`2662`, + :pr:`2672`). Any other message, a 9- to 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and @@ -174,7 +186,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. port's dissector, and :func:`~pcapkit.foundation.registry.protocols.transport.register_dccp_option` an option parser. The registries are :mod:`pcapkit.const.dccp`, crawled from - IANA. + IANA. Its ``make()`` refuses a value a field cannot hold with + :exc:`~pcapkit.utilities.exceptions.ProtocolError` naming the field + (:issue:`2614`, :pr:`2670`). * **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` @@ -325,7 +339,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. giving ``NO_PSEUDO_HEADER`` for a fragment's payload. GRE verifies only with its ``C`` bit set, and is ``NOT_COMPUTED`` with it clear; its ``make()`` takes ``checksum_present`` to set the bit and compute the checksum. GRE - states no length, so a frame the capture cut short reads as ``INVALID``. + states no length, so outside an IP layer, which states one, it takes the + captured and original lengths of its frame, and a GRE layer the capture cut + short is ``TRUNCATED`` either way (:issue:`2653`, :pr:`2668`). * **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, @@ -349,8 +365,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Added** -- :mod:`pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- ``beholding``, - ``frame_depth`` and ``rebuilding`` -- and importing nothing - from :mod:`pcapkit` (:issue:`2616`, :pr:`2623`). + ``frame_depth``, ``frame_length`` and ``rebuilding`` -- and importing + nothing from :mod:`pcapkit` (:issue:`2616`, :pr:`2623`). ``frame_length`` + holds the captured and original length of the frame being dissected, as its + PCAP record or PCAP-NG packet block states them, and is :data:`None` outside + a frame and inside an IP layer's payload (:issue:`2653`, :pr:`2668`). * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, @@ -391,6 +410,39 @@ HTTP/1 and HTTP/2 classes under the RFC names. unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. +* **Changed** -- **a breaking change to** building + :class:`~pcapkit.protocols.transport.tcp.TCP`, + :class:`~pcapkit.protocols.transport.udp.UDP` and + :class:`~pcapkit.protocols.transport.sctp.SCTP`: a header or option field + given an integer it cannot hold raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError` naming the field, where + 1.5.x wrote it masked to the field's width, and a fixed-size octet field + given the wrong number of octets raises, where it was padded or cut + (:issue:`2614`, :pr:`2670`). So ``TCP(window=70000)`` raises + ``TCP: invalid window: 70000``, where it wrote ``0x1170``; + ``TCP(seq_no=-1)`` and ``SCTP(vtag=-1)`` raise, where they wrote + ``0xFFFFFFFF``; a UDP payload of ``0xFFF8`` octets raises, where its Length + was written ``0``; and ``TCP(checksum=b'\x01')`` raises, where it wrote + ``0100``. The TCP User Timeout refuses a negative, ``True`` or non-numeric + timeout by name, where a negative one raised a bare :exc:`ValueError` and + ``True`` built one second. The Quick-Start TTL Diff, which :rfc:`4782` + defines modulo 256, wraps as in 1.5.x. + +* **Changed** -- **a breaking change to** building + :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` blocks: a field given a value + it cannot hold raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, + where 1.5.x wrote it masked (:issue:`2614`, :pr:`2671`). The Section Header + Block's Section Length takes ``-2**63`` to ``2**64 - 1``, ``-1`` and + ``0xFFFFFFFFFFFFFFFF`` writing the same octets, and refuses anything else, + where ``2**64`` wrote ``0``, ``True`` wrote ``1`` and ``None`` the ``-1`` + sentinel. The obsolete Packet Block's Drops Count takes ``0`` to ``0xFFFF``, + where ``-1`` wrote ``0xFFFF``, ``0x10000`` wrote ``0`` and ``True`` wrote + ``1``. The ``if_tzone`` and ``if_tsoffset`` options refuse a value outside + their signed 32 and 64 bits, where they wrote it masked, and the Interface + Description Block's snaplen refusal reads + ``PCAP-NG: invalid [IDB] snaplen: 4294967296``, where it read + ``PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296``. + * **Changed** -- **a breaking change to** :class:`~pcapkit.protocols.application.drarp.DRARP` and :class:`~pcapkit.protocols.link.inarp.InARP`, which move from @@ -498,6 +550,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. (:issue:`2614`, :pr:`2638`). A one-character :class:`str` given as flags, which passed the check and failed to pack, is refused the same way. +* **Changed** -- a protocol's ``__index__`` returns its value in the LinkType, + EtherType, TransType or AppType registry wherever one names it, and a class + an AppType member names returns that member's port as an :class:`int` + (:issue:`2627`, :pr:`2652`). So + :class:`~pcapkit.protocols.link.l2tpv2.L2TPv2`, + :class:`~pcapkit.protocols.application.ftp.FTP`, + :class:`~pcapkit.protocols.application.ftp_data.FTP_DATA`, the HTTP classes + and :class:`~pcapkit.protocols.application.ngap.NGAP`, which raised in 1.5.x, + return 1701, 21, 20, 80 and 38412, and TLS, SSLv3 and DTLS the ports TCP and + UDP dispatch them at, 443 and 5684; Radiotap, FRAD and GRE return their + LinkType or TransType member. SSLv2 and the SSL base, which nothing + dispatches to, raise as before, and no value 1.5.x returned changes. + * **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order (:issue:`2490`, :pr:`2491`, :pr:`2522`, :pr:`2523`, :pr:`2538`, @@ -554,7 +619,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. carry no ``version`` (:issue:`2628`, :pr:`2651`). The member-order convention's reference class is :class:`~pcapkit.protocols.link.arp.ARP`, where it was :class:`~pcapkit.protocols.link.ethernet.Ethernet`, whose MAC - address helpers moved to the base class (:issue:`2614`, :pr:`2638`). + address helpers moved to the base class (:issue:`2614`, :pr:`2638`). A + protocol's ``__index__`` returns its LinkType, EtherType, TransType or + AppType value ahead of any other registry's code, an AppType value as its + port (:issue:`2627`, :pr:`2652`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -587,6 +655,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``bot/issue-pr-kinds`` when it changes, never pushing ``main`` (:issue:`2561`, :pr:`2626`). +* **Changed** -- of the workflows, only Create Release waits for a deployment + approval: the ``pr`` jobs of the Issue and PR Kinds and Python Versions + workflows read the release App's key from the ``release-app`` environment, + which has no required reviewer, where they entered ``github-release`` and + waited on its approval, and skip with a warning while that environment + holds no key (:issue:`2648`, :pr:`2658`). + * **Changed** -- a pull request no longer adds its own changelog bullet: an unreleased version's entries are written centrally by the maintainer, and ``CONTRIBUTING.md`` and the pull request template say so (:issue:`2449`, @@ -640,6 +715,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. upper layer belonging to the enclosing IPv6 header (:issue:`2484`, :issue:`2487`, :pr:`2486`, :pr:`2488`). +* **Fixed** -- an HTTP/2 ``SETTINGS`` frame the capture cut short inside its + settings raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` from + the list of settings, so a segment carrying one was kept as + :class:`~pcapkit.protocols.misc.raw.Raw`. It parses as far as its octets go + and rebuilds as captured, and + :class:`~pcapkit.protocols.application.httpv2.HTTPv2`'s ``make()`` takes + ``frame_length``, from the frame's own length to ``0xFFFFFF``, so that the + declared Length survives (:issue:`2659`, :pr:`2666`). + * **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or socket crawler, without ``requests`` installed lost an entry of :data:`sys.path` on each failed attempt, so a :func:`pkgutil.walk_packages` From c86fa42f5f506e292f1356ed8e0f4860b90d5c04 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 14:53:12 -0500 Subject: [PATCH 16/27] docs(changelog): fold the internet makers' range checks and the cut HTTP/2 and SCTP rebuilds into 1.6.0 (#2449) - Breaking: the IPv4, HOPOPT, IPv6-Opts, IPv6-Route, MH and HIP makers (#2677) raise on a value a field cannot hold, where 1.5.x wrote it masked; a PDM scale past one octet raises where it warned, and MH._decode_signed is removed. The entry gives the old and new values. - Fixed: an SCTP packet cut short of its declared length rebuilds as captured (#2679). - Folded into unreleased entries: a cut HTTP/2 frame after the connection preface rebuilds, into the cut-SETTINGS entry (#2675); IPv4's Quick-Start function and not-used octet refusals, into the Quick-Start entry (#2677). - No entry for the test-only 42a61336e. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 ++++-- docs/source/changelog/1.6.0.rst | 50 +++++++++++++++++++++++++++++++-- 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 09bcde4cf1..857801e539 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,10 +46,12 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. -- **Changed** -- **a breaking change to** building a Quick-Start option in `IPv4`, `HOPOPT`, `IPv6_Opts` and `TCP`: a value its field cannot carry raises `ProtocolError`, where 1.5.x built it ([#2617](https://github.com/JarryShaw/PyPCAPKit/issues/2617), [#2630](https://github.com/JarryShaw/PyPCAPKit/pull/2630)). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and `True` as a TTL, nonce or reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. +- **Changed** -- **a breaking change to** building a Quick-Start option in `IPv4`, `HOPOPT`, `IPv6_Opts` and `TCP`: a value its field cannot carry raises `ProtocolError`, where 1.5.x built it ([#2617](https://github.com/JarryShaw/PyPCAPKit/issues/2617), [#2630](https://github.com/JarryShaw/PyPCAPKit/pull/2630)). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and `True` as a TTL, nonce or reserved value. IPv4's maker also refuses `True` as its function, and a Report of Approved Rate's not-used octet outside 0 to 255 ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. - **Changed** -- **a breaking change to** building `TCP`, `UDP` and `SCTP`: a header or option field given an integer it cannot hold raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and a fixed-size octet field given the wrong number of octets raises, where it was padded or cut ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). So `TCP(window=70000)` raises `TCP: invalid window: 70000`, where it wrote `0x1170`; `TCP(seq_no=-1)` and `SCTP(vtag=-1)` raise, where they wrote `0xFFFFFFFF`; a UDP payload of `0xFFF8` octets raises, where its Length was written `0`; and `TCP(checksum=b'\x01')` raises, where it wrote `0100`. The TCP User Timeout refuses a negative, `True` or non-numeric timeout by name, where a negative one raised a bare `ValueError` and `True` built one second. The Quick-Start TTL Diff, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) defines modulo 256, wraps as in 1.5.x. +- **Changed** -- **a breaking change to** building `IPv4`, `HOPOPT`, `IPv6_Opts`, `IPv6_Route`, `MH` and `HIP`, as TCP's, UDP's and SCTP's above: a header, option or parameter field given an integer it cannot hold, or `True` where it is wider than one bit, raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and so does a header too long for its length field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). So `IPv4(ttl=256)` raises `IPv4: invalid TTL: 256`, where it wrote `0`, and `IPv4(id=True)` raises, where it wrote `1`; a Binding Update's sequence number of `-1` raises `MH: invalid sequence number: -1`, where it wrote `0xFFFF`; and a HOPOPT or HIP header too long for its length field raises `HOPOPT: invalid Hdr Ext Len: 256` or `HIPv2: invalid header length: 256`, where that field was written `0`. A bit field raised `FieldValueError` for a value too wide and a bare `ValueError` or `OverflowError` for a negative one, where IPv4 options too long for the IHL, say, raise `IPv4: invalid IHL: 16`. A PDM scale past 255, given or needed for its delta, raises `HOPOPT: invalid PDM scale of delta time last received: 256` say, where it warned with `ProtocolWarning` and was written wrapped. HIP's lifetimes refuse `True`, which built as `1`, and a NaN or infinite one, which raised a bare `ValueError` or `OverflowError`. Where one of these checks replaces a refusal 1.5.x made, its message drops the option's or parameter's tag, as `HOPOPT: invalid RPL reserved flags: 32`, where it read `HOPOPT: [OptNo 99] invalid RPL reserved flags: 32`. `MH._decode_signed`, which decoded the geo-location degrees, is removed with no alias; the base class's `_read_sint` decodes them. + - **Changed** -- **a breaking change to** building `PCAPNG` blocks: a field given a value it cannot hold raises `ProtocolError`, where 1.5.x wrote it masked ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2671](https://github.com/JarryShaw/PyPCAPKit/pull/2671)). The Section Header Block's Section Length takes `-2**63` to `2**64 - 1`, `-1` and `0xFFFFFFFFFFFFFFFF` writing the same octets, and refuses anything else, where `2**64` wrote `0`, `True` wrote `1` and `None` the `-1` sentinel. The obsolete Packet Block's Drops Count takes `0` to `0xFFFF`, where `-1` wrote `0xFFFF`, `0x10000` wrote `0` and `True` wrote `1`. The `if_tzone` and `if_tsoffset` options refuse a value outside their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads `PCAP-NG: invalid [IDB] snaplen: 4294967296`, where it read `PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296`. - **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. @@ -98,7 +100,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). -- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses as far as its octets go and rebuilds as captured, and `HTTPv2`'s `make()` takes `frame_length`, from the frame's own length to `0xFFFFFF`, so that the declared Length survives ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666)). +- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses as far as its octets go and rebuilds as captured, and `HTTPv2`'s `make()` takes `frame_length`, from the frame's own length to `0xFFFFFF`, so that the declared Length survives ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). + +- **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 3341d6abe6..ad1503d4ab 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -404,7 +404,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. built it (:issue:`2617`, :pr:`2630`). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and ``True`` as a TTL, nonce or - reserved value. Refusing a TCP rate under 80 kbps reverses 1.5.0's + reserved value. IPv4's maker also refuses ``True`` as its function, and a + Report of Approved Rate's not-used octet outside 0 to 255 (:issue:`2614`, + :pr:`2677`). Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero (:issue:`1119`); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, @@ -428,6 +430,40 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``True`` built one second. The Quick-Start TTL Diff, which :rfc:`4782` defines modulo 256, wraps as in 1.5.x. +* **Changed** -- **a breaking change to** building + :class:`~pcapkit.protocols.internet.ipv4.IPv4`, + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT`, + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts`, + :class:`~pcapkit.protocols.internet.ipv6_route.IPv6_Route`, + :class:`~pcapkit.protocols.internet.mh.MH` and + :class:`~pcapkit.protocols.internet.hip.HIP`, as TCP's, UDP's and SCTP's + above: a header, option or parameter field given an integer it cannot hold, + or ``True`` where it is wider than one bit, raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError` naming the field, where + 1.5.x wrote it masked to the field's width, and so does a header too long + for its length field (:issue:`2614`, :pr:`2677`). So ``IPv4(ttl=256)`` + raises ``IPv4: invalid TTL: 256``, where it wrote ``0``, and + ``IPv4(id=True)`` raises, where it wrote ``1``; a Binding Update's sequence + number of ``-1`` raises ``MH: invalid sequence number: -1``, where it wrote + ``0xFFFF``; and a HOPOPT or HIP header too long for its length field raises + ``HOPOPT: invalid Hdr Ext Len: 256`` or + ``HIPv2: invalid header length: 256``, where that field was written ``0``. + A bit field raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` for + a value too wide and a bare :exc:`ValueError` or :exc:`OverflowError` for a + negative one, where IPv4 options too long for the IHL, say, raise + ``IPv4: invalid IHL: 16``. A PDM scale past + 255, given or needed for its delta, raises + ``HOPOPT: invalid PDM scale of delta time last received: 256`` say, where + it warned with :exc:`~pcapkit.utilities.warnings.ProtocolWarning` and was + written wrapped. HIP's lifetimes refuse ``True``, which built as ``1``, and + a NaN or infinite one, which raised a bare :exc:`ValueError` or + :exc:`OverflowError`. Where one of these checks replaces a refusal 1.5.x + made, its message drops the option's or parameter's tag, as + ``HOPOPT: invalid RPL reserved flags: 32``, where it read + ``HOPOPT: [OptNo 99] invalid RPL reserved flags: 32``. ``MH._decode_signed``, + which decoded the geo-location degrees, is removed with no alias; the base + class's ``_read_sint`` decodes them. + * **Changed** -- **a breaking change to** building :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` blocks: a field given a value it cannot hold raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, @@ -722,7 +758,17 @@ HTTP/1 and HTTP/2 classes under the RFC names. and rebuilds as captured, and :class:`~pcapkit.protocols.application.httpv2.HTTPv2`'s ``make()`` takes ``frame_length``, from the frame's own length to ``0xFFFFFF``, so that the - declared Length survives (:issue:`2659`, :pr:`2666`). + declared Length survives (:issue:`2659`, :pr:`2666`). A frame of any type + the capture cut short after the connection preface rebuilds as captured + too, where ``from_data`` gave the preface's first octets, as many as were + captured of the frame (:issue:`2665`, :pr:`2675`). + +* **Fixed** -- an :class:`~pcapkit.protocols.transport.sctp.SCTP` packet the + capture cut short of the length declared for it, by an IPv4 Total Length, + an IPv6 Payload Length or a caller's ``length``, rebuilt through + ``from_data`` without the octets after its last whole chunk, which its + header schema's ``bytes()`` counted twice. It rebuilds as captured, as one + parsed with no length declared did (:issue:`2673`, :pr:`2679`). * **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or socket crawler, without ``requests`` installed lost an entry of From 25659fd69b446dce02c43882853ccb75a6869941 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 17:27:20 -0500 Subject: [PATCH 17/27] docs(changelog): fold radiotap's field decoding, the MH and DSS length refusals and the Windows leg's argument file into 1.6.0 (#2449) - Breaking: MH(raw) and TCP(raw) (#2683) raise ProtocolError for an MH message whose Header Len is short of its fields, or a DSS option of a length its flags do not give, where 1.5.x read the gap as zeros and rebuilt it longer; a cut MH keeps its declared Header Len. The entry gives the old and new results, measured against v1.5.1. - Folded into unreleased entries: radiotap's decoded optional fields, Radiotap.__field__ with register_field and register_radiotap_field, the RadiotapField schemas and pcapkit.const.radiotap with its crawlers (#2657), and cut TLV items decoding zero-padded (#2690), into the Radiotap entry, with the IEEE 802.11 entry's Flags clause corrected; #2657's ruling on code to layout tables, into the conventions entry; the Windows leg's argument file (#2681), into the CI legs entry, as the cycle's other CI changes have entries. - No entry for the test-only 42a61336e. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 10 +++-- docs/source/changelog/1.6.0.rst | 73 +++++++++++++++++++++++++++++---- 2 files changed, 71 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 857801e539..00e2291906 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,9 +10,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)), as does an IP layer for IP protocol 47, through the registry (`Internet.__proto__`) IPv4 and IPv6 share ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). So with reassembly and tracing on, analysis follows a GRE tunnel: TCP inside GRE over IP reassembles under its inner addresses, and a fragmented GRE datagram reassembles to a parsed GRE layer. The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole rather than decoded. The 802.11 MAC frame behind them is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole as `fields`, which a rebuild writes, and decoded as `decoded`, an entry in wire order for each bit that selects a radiotap field or switches namespace, each field aligned from the start of the header as radiotap.org lays it out ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). That covers radiotap.org's defined fields, the RTS and data retries of bits 16 and 17, the radiotap and vendor namespace switches, a vendor namespace's data kept raw, and the items of TLV mode, a vendor item split into its header and raw data. An item a producer cut short of its layout, as radiotap.org allows, decodes with the octets it lacks as zeros and rebuilds as captured ([#2655](https://github.com/JarryShaw/PyPCAPKit/issues/2655), [#2690](https://github.com/JarryShaw/PyPCAPKit/pull/2690)). A bit no field is defined for, or a field running past the region, ends the walk as an `Unknown` entry holding the rest, and `make()` takes the fields decoded as well as raw. Each field type has a concrete `RadiotapField` schema, and is read and made through `Radiotap.__field__`, which `Radiotap.register_field` and `register_radiotap_field()` extend; `pcapkit.protocols.schema` and `pcapkit.protocols.data` export the schemas and data models with a `Radiotap_` prefix. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which keeps the bits it does not name. The field types and those registries are `pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag takes the name of its macro in Linux's `ieee80211_radiotap.h`, or in Wireshark where Linux defines none, and radiotap.org's wording where neither does. The 802.11 MAC frame behind the header is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). -- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11` (105) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` does not decode the Flags that mark them, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. +- **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11` (105) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` decodes the Flags that mark them but does not pass them on, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. @@ -44,6 +44,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. +- **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. + - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. - **Changed** -- **a breaking change to** building a Quick-Start option in `IPv4`, `HOPOPT`, `IPv6_Opts` and `TCP`: a value its field cannot carry raises `ProtocolError`, where 1.5.x built it ([#2617](https://github.com/JarryShaw/PyPCAPKit/issues/2617), [#2630](https://github.com/JarryShaw/PyPCAPKit/pull/2630)). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an IPv4 reserved value of 4 or more, and `True` as a TTL, nonce or reserved value. IPv4's maker also refuses `True` as its function, and a Report of Approved Rate's not-used octet outside 0 to 255 ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). Refusing a TCP rate under 80 kbps reverses 1.5.0's rounding of one down to zero ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)); a rate between two codes still rounds down to the lower. HOPOPT's and IPv6_Opts' Quick-Start maker builds an unassigned function code as the unassigned option their reader keeps it as, as IPv4's does. @@ -74,11 +76,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index ad1503d4ab..5398c38204 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -85,8 +85,34 @@ HTTP/1 and HTTP/2 classes under the RFC names. as do PCAP and PCAP-NG frames of ``LINKTYPE_IEEE802_11_RADIOTAP`` (127) (:issue:`2608`, :pr:`2632`). It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) and records every present-flags word, following bit 31 from one word to the - next. The optional fields those words select are kept whole rather than - decoded. The 802.11 MAC frame behind them is dissected as + next. The optional fields those words select are kept whole as ``fields``, + which a rebuild writes, and decoded as ``decoded``, an entry in wire order + for each bit that selects a radiotap field or switches namespace, each field + aligned from the start of the header as radiotap.org lays it out + (:issue:`2576`, :pr:`2657`). That covers radiotap.org's defined + fields, the RTS and data retries of bits 16 and 17, the radiotap and vendor + namespace switches, a vendor namespace's data kept raw, and the items of TLV + mode, a vendor item split into its header and raw data. An item a producer + cut short of its layout, as radiotap.org allows, decodes with the octets it + lacks as zeros and rebuilds as captured (:issue:`2655`, :pr:`2690`). A bit + no field is defined for, or a field running past the region, ends the walk + as an :class:`~pcapkit.protocols.data.link.radiotap.Unknown` entry holding + the rest, and ``make()`` takes the fields decoded as well as raw. Each field + type has a concrete + :class:`~pcapkit.protocols.schema.link.radiotap.RadiotapField` schema, and + is read and made through + :attr:`Radiotap.__field__ `, + which + :meth:`Radiotap.register_field ` + and :func:`~pcapkit.foundation.registry.protocols.link.register_radiotap_field` + extend; :mod:`pcapkit.protocols.schema` and :mod:`pcapkit.protocols.data` + export the schemas and data models with a ``Radiotap_`` prefix. A bitmap + radiotap.org defines bit by bit is a member of its flag registry, which + keeps the bits it does not name. The field types and those registries are + :mod:`pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag + takes the name of its macro in Linux's ``ieee80211_radiotap.h``, or in + Wireshark where Linux defines none, and radiotap.org's wording where neither + does. The 802.11 MAC frame behind the header is dissected as :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, below, looked up in ``Radiotap``'s own registry under :attr:`LinkType.IEEE802_11 `, @@ -102,10 +128,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, - with the FCS and radiotap's pad octets where captured, as ``Radiotap`` does - not decode the Flags that mark them, and any other protocol version is - refused. The frame types and subtypes are :mod:`pcapkit.const.ieee802_11`, - from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. + with the FCS and radiotap's pad octets where captured, as ``Radiotap`` + decodes the Flags that mark them but does not pass them on, and any other + protocol version is refused. The frame types and subtypes are + :mod:`pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std + 802.11-2020's Table 9-1. * **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` (:rfc:`792`) and :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` (:rfc:`4443`), on an @@ -386,6 +413,28 @@ HTTP/1 and HTTP/2 classes under the RFC names. is :data:`None` unless a caller supplies one; the ``Raw`` layer's ``info.error`` holds the failure instead. +* **Changed** -- **a breaking change to** parsing a Mobility Header or a + Multipath TCP DSS option whose length does not fit its fields: it raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x read the + fields it lacked as zeros, or dropped the octets past them, and rebuilt it + longer than captured (:issue:`2669`, :pr:`2683`). + :class:`~pcapkit.protocols.internet.mh.MH` refuses a whole message whose + Header Len is short of its type's fixed fields, a message + :rfc:`6275#section-9.2` has discarded, so a Home Test with Header Len 1 + raises ``MH: [Type 3] invalid header length: 1``, where its 24 octets + rebuilt as 32. Inside IPv6 it is the upper layer, kept as ``Raw`` by the + walk above, and the packet rebuilds as captured, where its 64 octets + rebuilt as 72. :class:`~pcapkit.protocols.transport.tcp.TCP` refuses a DSS + Length other than the one :rfc:`8684` gives its ``A``, ``a``, ``M`` and + ``m`` flags, or that plus the 2-octet checksum with ``M`` set, so a DSS of + Length 6 carrying a 4-octet Data ACK, which takes 8, raises + ``TCP: [OptNo 30] invalid format``, where its 40-octet segment rebuilt as + 44; inside IPv4 the segment is kept as + :class:`~pcapkit.protocols.misc.raw.Raw` and the packet rebuilds as + captured. A capture cut inside an MH message whose Header Len is short of + it rebuilds with the Header Len it declared, where 1.5.x wrote the one its + type takes. + * **Changed** -- **a breaking change to** construction: a checksum left out of ``make()`` is computed rather than zero, so the octets built change (:issue:`2471`, :pr:`2543`). :class:`~pcapkit.protocols.internet.ipv4.IPv4` @@ -658,7 +707,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. address helpers moved to the base class (:issue:`2614`, :pr:`2638`). A protocol's ``__index__`` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its - port (:issue:`2627`, :pr:`2652`). + port (:issue:`2627`, :pr:`2652`). A protocol's table from a code to the + layout read under it is a class-level registry map, with its + ``register_`` classmethod and wrapper, never a module-level + dictionary, and each layout is a concrete schema with typed fields, never + built inside the map (:issue:`2576`, :pr:`2657`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -677,7 +730,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``tests/protocols/internet``, ``tests/protocols/application`` and ``tests/protocols/misc`` as shards of their own ahead of the rest of ``tests/protocols``, which run whole outgrew the runner's memory and - stalled at its tail (:issue:`2600`, :pr:`2604`). + stalled at its tail (:issue:`2600`, :pr:`2604`). The experimental + workflow's Windows leg passes pytest the modules it ignores as an + ``@``-prefixed argument file, where they went on its command line and had + outgrown Windows' limit, so the leg failed before pytest started + (:issue:`2678`, :pr:`2681`). * **Changed** -- once a release's milestone is closed while ``__version__`` still names that release, the milestone workflow gives an item the lowest From 285a8380041b6ab1aecb27470d830978ac6a374b Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 18:56:34 -0500 Subject: [PATCH 18/27] docs(changelog): fold the MH option refusals, radiotap's mixed bitmaps, the cut schema lists, the TSO checksum cut and short ZigBee keys into 1.6.0 (#2449) - Breaking: MH (#2700) refuses a mobility option, or a sub-option, QoS attribute or CGA extension of one, whose fields do not take its length, where 1.5.x rebuilt another; QualityOfServiceOption gains a required tc_reserved. Prefix lengths are held to their field's width, the NAI and service identifier makers count octets, and a long-form DER CGA key parses. Folded into the MH/DSS and makers' range-check entries, with a QoS entry of its own; old and new values measured against v1.5.1. - Fixed: ListField zero-fills a cut list of schema items, bounded by ZERO_PAD_BUDGET_RATIO with no free band, and a SACK past the data raises ProtocolError where it raised FieldValueError (#2701); a Decryption Secrets Block made from a short parsed ZigBee key is written as captured, and zigbee_key_held's schema holds the octets given (#2697). - Folded into unreleased entries: radiotap's mixed bitmaps, their known-bits registries and BitmapField (#2696), into the Radiotap entry; the TSO and AH checksum cut (#2699), into the checksum and contextvars entries; the zero-filled SETTINGS and IGMPv3 records and the IGMPv3 Additional Data rebuild (#2701), into the HTTP/2 SETTINGS and IGMP entries, dropping HTTPv2.make's unreleased frame_length. - No entry for the docstring-and-test-only be8306140 (#2695). - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 20 ++++-- docs/source/changelog/1.6.0.rst | 118 +++++++++++++++++++++++++++----- 2 files changed, 112 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 00e2291906..efa22dd3ac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,13 +10,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)), as does an IP layer for IP protocol 47, through the registry (`Internet.__proto__`) IPv4 and IPv6 share ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). So with reassembly and tracing on, analysis follows a GRE tunnel: TCP inside GRE over IP reassembles under its inner addresses, and a fragmented GRE datagram reassembles to a parsed GRE layer. The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole as `fields`, which a rebuild writes, and decoded as `decoded`, an entry in wire order for each bit that selects a radiotap field or switches namespace, each field aligned from the start of the header as radiotap.org lays it out ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). That covers radiotap.org's defined fields, the RTS and data retries of bits 16 and 17, the radiotap and vendor namespace switches, a vendor namespace's data kept raw, and the items of TLV mode, a vendor item split into its header and raw data. An item a producer cut short of its layout, as radiotap.org allows, decodes with the octets it lacks as zeros and rebuilds as captured ([#2655](https://github.com/JarryShaw/PyPCAPKit/issues/2655), [#2690](https://github.com/JarryShaw/PyPCAPKit/pull/2690)). A bit no field is defined for, or a field running past the region, ends the walk as an `Unknown` entry holding the rest, and `make()` takes the fields decoded as well as raw. Each field type has a concrete `RadiotapField` schema, and is read and made through `Radiotap.__field__`, which `Radiotap.register_field` and `register_radiotap_field()` extend; `pcapkit.protocols.schema` and `pcapkit.protocols.data` export the schemas and data models with a `Radiotap_` prefix. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which keeps the bits it does not name. The field types and those registries are `pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag takes the name of its macro in Linux's `ieee80211_radiotap.h`, or in Wireshark where Linux defines none, and radiotap.org's wording where neither does. The 802.11 MAC frame behind the header is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole as `fields`, which a rebuild writes, and decoded as `decoded`, an entry in wire order for each bit that selects a radiotap field or switches namespace, each field aligned from the start of the header as radiotap.org lays it out ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). That covers radiotap.org's defined fields, the RTS and data retries of bits 16 and 17, the radiotap and vendor namespace switches, a vendor namespace's data kept raw, and the items of TLV mode, a vendor item split into its header and raw data. An item a producer cut short of its layout, as radiotap.org allows, decodes with the octets it lacks as zeros and rebuilds as captured ([#2655](https://github.com/JarryShaw/PyPCAPKit/issues/2655), [#2690](https://github.com/JarryShaw/PyPCAPKit/pull/2690)). A bit no field is defined for, or a field running past the region, ends the walk as an `Unknown` entry holding the rest, and `make()` takes the fields decoded as well as raw. Each field type has a concrete `RadiotapField` schema, and is read and made through `Radiotap.__field__`, which `Radiotap.register_field` and `register_radiotap_field()` extend; `pcapkit.protocols.schema` and `pcapkit.protocols.data` export the schemas and data models with a `Radiotap_` prefix. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which keeps the bits it does not name. One mixing value subfields with flags -- MCS `flags`, HE `data1` and `data2`, HE-MU `flags1` and `flags2` and U-SIG `common` -- decodes into a data model of its subfields, each an `int`, and, except in MCS's, its other bits as `known`, a member of a known-bits registry such as `HEData1Known`; `make()` takes it as a mapping, refusing a subfield its bits cannot hold or a `known` setting a subfield's bit with `ProtocolError` ([#2647](https://github.com/JarryShaw/PyPCAPKit/issues/2647), [#2696](https://github.com/JarryShaw/PyPCAPKit/pull/2696)). `BitmapField` reads such a word, an integer in its byte order split into subfields counted from its least-significant bit. The field types and those registries are `pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag takes the name of its macro in Linux's `ieee80211_radiotap.h`, or in Wireshark where Linux defines none, and radiotap.org's wording where neither does. The 802.11 MAC frame behind the header is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). - **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11` (105) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` decodes the Flags that mark them but does not pass them on, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses as far as its octets go and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). The octets it lacks read as zeros, the group records included, and a Report's header schema cut inside the Additional Data past its records packs back as captured ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2687](https://github.com/JarryShaw/PyPCAPKit/issues/2687), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. - **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. HTTP's base follows the same rules, a breaking change given below ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. @@ -34,17 +34,19 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). ICMP's, IGMP's and GRE's checksums take those lengths as well behind an IPv4 Total Length of 0, as TCP segmentation offload leaves it, and behind an AH there, so one the capture cut short is `TRUNCATED` with no warning; a GRE or IGMP layer read directly from octets that end inside its header or message is `TRUNCATED` too ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). - **Added** -- `pcapkit.corekit.fields.strings.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)). -- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `frame_length` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). `frame_length` holds the captured and original length of the frame being dissected, as its PCAP record or PCAP-NG packet block states them, and is `None` outside a frame and inside an IP layer's payload ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). +- **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `frame_length` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). `frame_length` holds the captured and original length of the frame being dissected, as its PCAP record or PCAP-NG packet block states them, and is `None` outside a frame and inside an IP layer's payload ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)), unless the layer states no length for it: an IPv4 Total Length of 0, or an AH that IPv4 carries, hands the frame's lengths on ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. -- **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. +- **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. A mobility option whose fields do not take its Length is refused too, and so are such a sub-option, QoS attribute or CGA extension inside one, a target care-of address neither IPv4 nor IPv6, and CGA Parameters whose fixed fields or public key run past the option, where 1.5.x read octets that were not the option's and rebuilt another ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). So an MN-ID option of Length 0 raises `MH: [OptNo 8] invalid option length: 0`, where 1.5.x read its fields from the octets after it and rebuilt it with Length 17, and a Flow Identification option's 8-octet target care-of address raises `MH: [FID SubOpt 5] invalid sub-option length: 10`, where 1.5.x parsed it and raised `FieldValueError` rebuilding it; inside IPv6 the header is `Raw` and the packet rebuilds as captured. + +- **Changed** -- **a breaking change to** `QualityOfServiceOption`, an MH Quality-of-Service option's data model, which gains a required `tc_reserved` after `dscp`: the two reserved bits of the traffic class, which 1.5.x dropped, so a traffic class of `0xB9` rebuilt as `0xB8` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Building the model without it raises `TypeError`, and `MH`'s `make()` takes it, `0` by default. The MN-ID and Service Selection makers count the octets an identifier packs as, so an NAI of `'usér@x'` takes Length 8, where 1.5.x wrote 7 from its characters, and a service identifier parsed from octets that are not UTF-8 rebuilds as captured, where 1.5.x re-encoded its text, under Length 4 where 3 were captured. A CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses through the long form of its DER length, where 1.5.x misread the length and raised `ProtocolError`. - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. @@ -52,7 +54,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** building `TCP`, `UDP` and `SCTP`: a header or option field given an integer it cannot hold raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and a fixed-size octet field given the wrong number of octets raises, where it was padded or cut ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). So `TCP(window=70000)` raises `TCP: invalid window: 70000`, where it wrote `0x1170`; `TCP(seq_no=-1)` and `SCTP(vtag=-1)` raise, where they wrote `0xFFFFFFFF`; a UDP payload of `0xFFF8` octets raises, where its Length was written `0`; and `TCP(checksum=b'\x01')` raises, where it wrote `0100`. The TCP User Timeout refuses a negative, `True` or non-numeric timeout by name, where a negative one raised a bare `ValueError` and `True` built one second. The Quick-Start TTL Diff, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) defines modulo 256, wraps as in 1.5.x. -- **Changed** -- **a breaking change to** building `IPv4`, `HOPOPT`, `IPv6_Opts`, `IPv6_Route`, `MH` and `HIP`, as TCP's, UDP's and SCTP's above: a header, option or parameter field given an integer it cannot hold, or `True` where it is wider than one bit, raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and so does a header too long for its length field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). So `IPv4(ttl=256)` raises `IPv4: invalid TTL: 256`, where it wrote `0`, and `IPv4(id=True)` raises, where it wrote `1`; a Binding Update's sequence number of `-1` raises `MH: invalid sequence number: -1`, where it wrote `0xFFFF`; and a HOPOPT or HIP header too long for its length field raises `HOPOPT: invalid Hdr Ext Len: 256` or `HIPv2: invalid header length: 256`, where that field was written `0`. A bit field raised `FieldValueError` for a value too wide and a bare `ValueError` or `OverflowError` for a negative one, where IPv4 options too long for the IHL, say, raise `IPv4: invalid IHL: 16`. A PDM scale past 255, given or needed for its delta, raises `HOPOPT: invalid PDM scale of delta time last received: 256` say, where it warned with `ProtocolWarning` and was written wrapped. HIP's lifetimes refuse `True`, which built as `1`, and a NaN or infinite one, which raised a bare `ValueError` or `OverflowError`. Where one of these checks replaces a refusal 1.5.x made, its message drops the option's or parameter's tag, as `HOPOPT: invalid RPL reserved flags: 32`, where it read `HOPOPT: [OptNo 99] invalid RPL reserved flags: 32`. `MH._decode_signed`, which decoded the geo-location degrees, is removed with no alias; the base class's `_read_sint` decodes them. +- **Changed** -- **a breaking change to** building `IPv4`, `HOPOPT`, `IPv6_Opts`, `IPv6_Route`, `MH` and `HIP`, as TCP's, UDP's and SCTP's above: a header, option or parameter field given an integer it cannot hold, or `True` where it is wider than one bit, raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and so does a header too long for its length field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). So `IPv4(ttl=256)` raises `IPv4: invalid TTL: 256`, where it wrote `0`, and `IPv4(id=True)` raises, where it wrote `1`; a Binding Update's sequence number of `-1` raises `MH: invalid sequence number: -1`, where it wrote `0xFFFF`; and a HOPOPT or HIP header too long for its length field raises `HOPOPT: invalid Hdr Ext Len: 256` or `HIPv2: invalid header length: 256`, where that field was written `0`. A bit field raised `FieldValueError` for a value too wide and a bare `ValueError` or `OverflowError` for a negative one, where IPv4 options too long for the IHL, say, raise `IPv4: invalid IHL: 16`. A PDM scale past 255, given or needed for its delta, raises `HOPOPT: invalid PDM scale of delta time last received: 256` say, where it warned with `ProtocolWarning` and was written wrapped. HIP's lifetimes refuse `True`, which built as `1`, and a NaN or infinite one, which raised a bare `ValueError` or `OverflowError`. An MH prefix length is held to its field's width alone, as a parse keeps it -- 0 to 63 in the IPv4 Address Acknowledgement and the IPv4 Home Address Request and Reply, 0 to 255 in the Delegated Mobile Network Prefix -- so a 33 builds and a parsed 63 rebuilds, where 1.5.x refused one past the address, as `MH: [OptNo 30] invalid prefix length: 33` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Where one of these checks replaces a refusal 1.5.x made, its message drops the option's or parameter's tag, as `HOPOPT: invalid RPL reserved flags: 32`, where it read `HOPOPT: [OptNo 99] invalid RPL reserved flags: 32`. `MH._decode_signed`, which decoded the geo-location degrees, is removed with no alias; the base class's `_read_sint` decodes them. - **Changed** -- **a breaking change to** building `PCAPNG` blocks: a field given a value it cannot hold raises `ProtocolError`, where 1.5.x wrote it masked ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2671](https://github.com/JarryShaw/PyPCAPKit/pull/2671)). The Section Header Block's Section Length takes `-2**63` to `2**64 - 1`, `-1` and `0xFFFFFFFFFFFFFFFF` writing the same octets, and refuses anything else, where `2**64` wrote `0`, `True` wrote `1` and `None` the `-1` sentinel. The obsolete Packet Block's Drops Count takes `0` to `0xFFFF`, where `-1` wrote `0xFFFF`, `0x10000` wrote `0` and `True` wrote `1`. The `if_tzone` and `if_tsoffset` options refuse a value outside their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads `PCAP-NG: invalid [IDB] snaplen: 4294967296`, where it read `PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296`. @@ -102,10 +104,14 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). -- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses as far as its octets go and rebuilds as captured, and `HTTPv2`'s `make()` takes `frame_length`, from the frame's own length to `0xFFFFFF`, so that the declared Length survives ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). +- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside zero-filled to its length, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Each item filled is a schema object rather than a zero octet, so `ZERO_PAD_BUDGET_RATIO` bounds the fill from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. + +- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses, with the octets its settings lack read as zeros, as a cut `DATA` frame's data is, and rebuilds as captured ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). - **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). +- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). + - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). --- diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 5398c38204..6ee171527e 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -108,7 +108,18 @@ HTTP/1 and HTTP/2 classes under the RFC names. extend; :mod:`pcapkit.protocols.schema` and :mod:`pcapkit.protocols.data` export the schemas and data models with a ``Radiotap_`` prefix. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which - keeps the bits it does not name. The field types and those registries are + keeps the bits it does not name. One mixing value subfields with flags -- + MCS ``flags``, HE ``data1`` and ``data2``, HE-MU ``flags1`` and ``flags2`` + and U-SIG ``common`` -- decodes into a data model of its subfields, each an + :class:`int`, and, except in MCS's, its other bits as ``known``, a member of a + known-bits registry such as + :class:`~pcapkit.const.radiotap.he_data1_known.HEData1Known`; ``make()`` + takes it as a mapping, refusing a subfield its bits cannot hold or a + ``known`` setting a subfield's bit with + :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2647`, + :pr:`2696`). :class:`~pcapkit.corekit.fields.numbers.BitmapField` reads + such a word, an integer in its byte order split into subfields counted from + its least-significant bit. The field types and those registries are :mod:`pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag takes the name of its macro in Linux's ``ieee80211_radiotap.h``, or in Wireshark where Linux defines none, and radiotap.org's wording where neither @@ -160,11 +171,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. :pr:`2444`, :pr:`2465`). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the - capture cut short, a Report cut inside its group records included, parses as - far as its octets go and rebuilds as captured, whether parsed on its own or - carried by IPv4, and IGMPv3's ``make()`` takes ``num_group_records``, from - the record count to ``0xFFFF`` (:issue:`2650`, :issue:`2660`, :pr:`2662`, - :pr:`2672`). Any other message, a 9- to + capture cut short, a Report cut inside its group records included, parses + and rebuilds as captured, whether parsed on its own or carried by IPv4, and + IGMPv3's ``make()`` takes ``num_group_records``, from the record count to + ``0xFFFF`` (:issue:`2650`, :issue:`2660`, :pr:`2662`, :pr:`2672`). The + octets it lacks read as zeros, the group records included, and a Report's + header schema cut inside the Additional Data past its records packs back as + captured (:issue:`2686`, :issue:`2687`, :pr:`2701`). Any other message, a 9- to 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and @@ -368,7 +381,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. takes ``checksum_present`` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut - short is ``TRUNCATED`` either way (:issue:`2653`, :pr:`2668`). + short is ``TRUNCATED`` either way (:issue:`2653`, :pr:`2668`). ICMP's, IGMP's + and GRE's checksums take those lengths as well behind an IPv4 Total Length + of 0, as TCP segmentation offload leaves it, and behind an AH there, so one + the capture cut short is ``TRUNCATED`` with no warning; a GRE or IGMP layer + read directly from octets that end inside its header or message is + ``TRUNCATED`` too (:issue:`2689`, :pr:`2699`). * **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, @@ -396,7 +414,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. nothing from :mod:`pcapkit` (:issue:`2616`, :pr:`2623`). ``frame_length`` holds the captured and original length of the frame being dissected, as its PCAP record or PCAP-NG packet block states them, and is :data:`None` outside - a frame and inside an IP layer's payload (:issue:`2653`, :pr:`2668`). + a frame and inside an IP layer's payload (:issue:`2653`, :pr:`2668`), unless + the layer states no length for it: an IPv4 Total Length of 0, or an AH that + IPv4 carries, hands the frame's lengths on (:issue:`2689`, :pr:`2699`). * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, @@ -433,7 +453,35 @@ HTTP/1 and HTTP/2 classes under the RFC names. :class:`~pcapkit.protocols.misc.raw.Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its - type takes. + type takes. A mobility option whose fields do not take its Length is + refused too, and so are such a sub-option, QoS attribute or CGA extension + inside one, a target care-of address neither IPv4 nor IPv6, and CGA + Parameters whose fixed fields or public key run past the option, where + 1.5.x read octets that were not the option's and rebuilt another + (:issue:`2684`, :pr:`2700`). So an MN-ID option of Length 0 raises + ``MH: [OptNo 8] invalid option length: 0``, where 1.5.x read its fields from + the octets after it and rebuilt it with Length 17, and a Flow Identification + option's 8-octet target care-of address raises + ``MH: [FID SubOpt 5] invalid sub-option length: 10``, + where 1.5.x parsed it and raised + :exc:`~pcapkit.utilities.exceptions.FieldValueError` rebuilding it; inside + IPv6 the header is ``Raw`` and the packet rebuilds as captured. + +* **Changed** -- **a breaking change to** + :class:`~pcapkit.protocols.data.internet.mh.QualityOfServiceOption`, an MH + Quality-of-Service option's data model, which gains a required + ``tc_reserved`` after ``dscp``: the two reserved bits of the traffic class, + which 1.5.x dropped, so a traffic class of ``0xB9`` rebuilt as ``0xB8`` + (:issue:`2684`, :pr:`2700`). Building the model without it raises + :exc:`TypeError`, and :class:`~pcapkit.protocols.internet.mh.MH`'s + ``make()`` takes it, ``0`` by default. The MN-ID and Service Selection + makers count the octets an identifier packs as, so an NAI of ``'usér@x'`` + takes Length 8, where 1.5.x wrote 7 from its characters, and a service + identifier parsed from octets that are not UTF-8 rebuilds as captured, + where 1.5.x re-encoded its text, under Length 4 where 3 were captured. A + CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses + through the long form of its DER length, where 1.5.x misread the length + and raised :exc:`~pcapkit.utilities.exceptions.ProtocolError`. * **Changed** -- **a breaking change to** construction: a checksum left out of ``make()`` is computed rather than zero, so the octets built change @@ -506,8 +554,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. it warned with :exc:`~pcapkit.utilities.warnings.ProtocolWarning` and was written wrapped. HIP's lifetimes refuse ``True``, which built as ``1``, and a NaN or infinite one, which raised a bare :exc:`ValueError` or - :exc:`OverflowError`. Where one of these checks replaces a refusal 1.5.x - made, its message drops the option's or parameter's tag, as + :exc:`OverflowError`. An MH prefix length is held to its field's width + alone, as a parse keeps it -- 0 to 63 in the IPv4 Address Acknowledgement + and the IPv4 Home Address Request and Reply, 0 to 255 in the Delegated + Mobile Network Prefix -- so a 33 builds and a parsed 63 rebuilds, where + 1.5.x refused one past the address, as + ``MH: [OptNo 30] invalid prefix length: 33`` (:issue:`2684`, :pr:`2700`). + Where one of these checks replaces a refusal 1.5.x made, its message drops + the option's or parameter's tag, as ``HOPOPT: invalid RPL reserved flags: 32``, where it read ``HOPOPT: [OptNo 99] invalid RPL reserved flags: 32``. ``MH._decode_signed``, which decoded the geo-location degrees, is removed with no alias; the base @@ -808,17 +862,33 @@ HTTP/1 and HTTP/2 classes under the RFC names. upper layer belonging to the enclosing IPv6 header (:issue:`2484`, :issue:`2487`, :pr:`2486`, :pr:`2488`). +* **Fixed** -- :meth:`ListField.unpack ` + reads a list of schema items the data ends inside zero-filled to its length, + as :meth:`FieldBase.unpack ` + reads any other field, so a schema cut inside one packs back as captured, + where the first item past the data raised + :exc:`~pcapkit.utilities.exceptions.FieldValueError` for consuming nothing + (:issue:`2686`, :pr:`2701`). Each item filled is a schema object rather than + a zero octet, so :data:`~pcapkit.utilities.config.ZERO_PAD_BUDGET_RATIO` + bounds the fill from its first octet, with neither the free band of + :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_SHORTFALL` nor a one-off + allowance, and a cut list declaring more than + :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_LENGTH` is refused. A TCP + ``SACK`` option running past the data raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError`, + ``Field options has an option that runs past the end of the data: ...``, + where it raised that ``FieldValueError``, + ``Field sack has an item that consumed no data: ...``. + * **Fixed** -- an HTTP/2 ``SETTINGS`` frame the capture cut short inside its settings raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` from the list of settings, so a segment carrying one was kept as - :class:`~pcapkit.protocols.misc.raw.Raw`. It parses as far as its octets go - and rebuilds as captured, and - :class:`~pcapkit.protocols.application.httpv2.HTTPv2`'s ``make()`` takes - ``frame_length``, from the frame's own length to ``0xFFFFFF``, so that the - declared Length survives (:issue:`2659`, :pr:`2666`). A frame of any type - the capture cut short after the connection preface rebuilds as captured - too, where ``from_data`` gave the preface's first octets, as many as were - captured of the frame (:issue:`2665`, :pr:`2675`). + :class:`~pcapkit.protocols.misc.raw.Raw`. It parses, with the octets its + settings lack read as zeros, as a cut ``DATA`` frame's data is, and rebuilds + as captured (:issue:`2659`, :issue:`2686`, :pr:`2666`, :pr:`2701`). A frame + of any type the capture cut short after the connection preface rebuilds as + captured too, where ``from_data`` gave the preface's first octets, as many + as were captured of the frame (:issue:`2665`, :pr:`2675`). * **Fixed** -- an :class:`~pcapkit.protocols.transport.sctp.SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, @@ -827,6 +897,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. header schema's ``bytes()`` counted twice. It rebuilds as captured, as one parsed with no length declared did (:issue:`2673`, :pr:`2679`). +* **Fixed** -- a :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` Decryption + Secrets Block that ``make()`` built from a parsed ZigBee NWK or APS key + shorter than its fixed width was corrupt: it declared that width, 18 or 22 + octets, as its Secrets Length and sized the block from it, but wrote only + the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, + where 28 were captured. It is written as captured, and + ``pcapkit.protocols.schema.misc.pcapng.zigbee_key_held`` returns a schema + whose :func:`bytes` and :func:`len` are those of the octets held, where they + were the fixed width's (:issue:`2688`, :pr:`2697`). + * **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or socket crawler, without ``requests`` installed lost an entry of :data:`sys.path` on each failed attempt, so a :func:`pkgutil.walk_packages` From d2aee5bef2a8523ae62965ff413f03c70fdb8c71 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 22:24:57 -0500 Subject: [PATCH 19/27] docs(changelog): fold make()'s enumeration fields, the corekit.schema helpers, MH options 6 and 27, the cut SCTP and option-area schemas and dict-built secrets into 1.6.0 (#2449) - Breaking: a make() enumeration field takes a member or an int, its X_default/X_namespace/X_reversed keywords removed with no alias, and names, wide codes and wide option codes are refused (#2692); the schema helpers move to pcapkit.corekit.schema and pcapkit.corekit.fields.byteorder is removed (#2704); MobileNetworkPrefixOption gains prefix_length and keeps an IPv6Address prefix, and TimestampOption's timestamp is None past 9999 (#2709). Folded into the MH data-model entry; old values measured against v1.5.1. - Fixed: the SCTP schema cut inside its padding and a header schema cut inside an option area pack back as captured (#2682), into the SCTP entry; a Decryption Secrets Block rebuilds from its dict as captured (#2707), into the DSB entry. - Folded into unreleased entries: FRAD's Q.922 address refusal and quiet cut rebuild (#2719); a from_data rebuild's checksum status (#2712); a failed ListField fill's ledger (#2706); pad_length's path (#2704); TLS's enumeration fields (#2692); the make() and schema-helper rulings, into the conventions entry (#2692, #2704); the GraalPy shards, into the CI-legs entry (#2720). - 1.5.0's corekit.fields.byteorder role points at corekit.schema.byteorder, where #2704 moved it, so the nitpicky build does not warn on it; 1.5.0's text unchanged. - No entry for the test-only 72b2bd83f (#2715), or the docs-and-test-only 0c0794097 (#2718). - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 24 +++--- docs/source/changelog/1.5.0.rst | 6 +- docs/source/changelog/1.6.0.rst | 134 ++++++++++++++++++++++++++++---- 3 files changed, 135 insertions(+), 29 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index efa22dd3ac..c065e9b13a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)), as does a protocol class declaring `code=ARPHRD.X`, since `ARPHRD` names `LinuxSLL` as its `__proto_dest__` ([#2533](https://github.com/JarryShaw/PyPCAPKit/issues/2533), [#2598](https://github.com/JarryShaw/PyPCAPKit/pull/2598)). A parse told to stop there -- at `LinuxSLL`, the version itself or the link layer -- keeps the payload `Raw` on either path ([#2532](https://github.com/JarryShaw/PyPCAPKit/issues/2532), [#2587](https://github.com/JarryShaw/PyPCAPKit/pull/2587)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)), and `pcapkit.const` exports it as `Linux_ARPHRD` ([#2661](https://github.com/JarryShaw/PyPCAPKit/issues/2661), [#2663](https://github.com/JarryShaw/PyPCAPKit/pull/2663)). -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)), as do PCAP and PCAP-NG frames of `LINKTYPE_FRELAY` (107) ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)), as do PCAP and PCAP-NG frames of `LINKTYPE_FRELAY` (107) ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). A Q.922 address other than the 2-octet form, its EA bits other than 0 then 1, is refused with `ProtocolError` unless the capture cut it short, and is kept as `Raw` with nothing logged behind `LinuxSLL` or a `LINKTYPE_FRELAY` capture. A header the capture cut short, before its NLPID included, rebuilds as captured and logs nothing ([#2710](https://github.com/JarryShaw/PyPCAPKit/issues/2710), [#2719](https://github.com/JarryShaw/PyPCAPKit/pull/2719)). - **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as `Raw` unless `register_netlink()` registers a dissector for its `nlmsg_type` ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). @@ -22,7 +22,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. Its `make()` refuses a value a field cannot hold with `ProtocolError` naming the field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. `pcapkit.protocols.data` and `pcapkit.protocols.schema` export a `tls12_cid` record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)), and a name, or a code too wide, for an enumeration field with `ProtocolError`, as the other makers do, below ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. `pcapkit.protocols.data` and `pcapkit.protocols.schema` export a `tls12_cid` record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. @@ -34,11 +34,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). ICMP's, IGMP's and GRE's checksums take those lengths as well behind an IPv4 Total Length of 0, as TCP segmentation offload leaves it, and behind an AH there, so one the capture cut short is `TRUNCATED` with no warning; a GRE or IGMP layer read directly from octets that end inside its header or message is `TRUNCATED` too ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). ICMP's, IGMP's and GRE's checksums take those lengths as well behind an IPv4 Total Length of 0, as TCP segmentation offload leaves it, and behind an AH there, so one the capture cut short is `TRUNCATED` with no warning; a GRE or IGMP layer read directly from octets that end inside its header or message is `TRUNCATED` too ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). A layer `from_data` rebuilds from a cut capture, from its data model or its `dict`, verifies as its parse did, so a cut GRE, ICMP, IGMP or OSPF layer is `TRUNCATED` with no warning ([#2703](https://github.com/JarryShaw/PyPCAPKit/issues/2703), [#2712](https://github.com/JarryShaw/PyPCAPKit/pull/2712)). - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). -- **Added** -- `pcapkit.corekit.fields.strings.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)). +- **Added** -- `pcapkit.corekit.schema.padding.pad_length()`, the octets of padding that take a length to an `align`-octet boundary, and a `PaddingField` sized by it, given `align` and `after` -- the boundary, and the field or callable whose length the padding follows -- in place of `length` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2624](https://github.com/JarryShaw/PyPCAPKit/pull/2624)), in the `pcapkit.corekit.schema` package of helpers shared across the schema layer, given below ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). - **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `frame_length` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). `frame_length` holds the captured and original length of the frame being dissected, as its PCAP record or PCAP-NG packet block states them, and is `None` outside a frame and inside an IP layer's payload ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)), unless the layer states no length for it: an IPv4 Total Length of 0, or an AH that IPv4 carries, hands the frame's lengths on ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). @@ -46,7 +46,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. A mobility option whose fields do not take its Length is refused too, and so are such a sub-option, QoS attribute or CGA extension inside one, a target care-of address neither IPv4 nor IPv6, and CGA Parameters whose fixed fields or public key run past the option, where 1.5.x read octets that were not the option's and rebuilt another ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). So an MN-ID option of Length 0 raises `MH: [OptNo 8] invalid option length: 0`, where 1.5.x read its fields from the octets after it and rebuilt it with Length 17, and a Flow Identification option's 8-octet target care-of address raises `MH: [FID SubOpt 5] invalid sub-option length: 10`, where 1.5.x parsed it and raised `FieldValueError` rebuilding it; inside IPv6 the header is `Raw` and the packet rebuilds as captured. -- **Changed** -- **a breaking change to** `QualityOfServiceOption`, an MH Quality-of-Service option's data model, which gains a required `tc_reserved` after `dscp`: the two reserved bits of the traffic class, which 1.5.x dropped, so a traffic class of `0xB9` rebuilt as `0xB8` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Building the model without it raises `TypeError`, and `MH`'s `make()` takes it, `0` by default. The MN-ID and Service Selection makers count the octets an identifier packs as, so an NAI of `'usér@x'` takes Length 8, where 1.5.x wrote 7 from its characters, and a service identifier parsed from octets that are not UTF-8 rebuilds as captured, where 1.5.x re-encoded its text, under Length 4 where 3 were captured. A CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses through the long form of its DER length, where 1.5.x misread the length and raised `ProtocolError`. +- **Changed** -- **a breaking change to** MH option data models. `QualityOfServiceOption`, the Quality-of-Service option's, gains a required `tc_reserved` after `dscp`: the two reserved bits of the traffic class, which 1.5.x dropped, so a traffic class of `0xB9` rebuilt as `0xB8` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Building the model without it raises `TypeError`, and `MH`'s `make()` takes it, `0` by default. The MN-ID and Service Selection makers count the octets an identifier packs as, so an NAI of `'usér@x'` takes Length 8, where 1.5.x wrote 7 from its characters, and a service identifier parsed from octets that are not UTF-8 rebuilds as captured, where 1.5.x re-encoded its text, under Length 4 where 3 were captured. A CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses through the long form of its DER length, where 1.5.x misread the length and raised `ProtocolError`. `MobileNetworkPrefixOption`, the Mobile Network Prefix option's, gains a required `prefix_length`, and its `prefix` is the `IPv6Address` captured, where it was an `IPv6Network`, so a prefix with bits set past its length, or a length past 128, parses and rebuilds, where 1.5.x raised a bare `ValueError` such as `2001:db8::1/64 has host bits set` ([#2702](https://github.com/JarryShaw/PyPCAPKit/issues/2702), [#2709](https://github.com/JarryShaw/PyPCAPKit/pull/2709)). Its maker takes `prefix_length`, from 0 to 255, where 1.5.x ignored it and wrote the length `prefix` gave, 128 for an address; and it takes `prefix` as an address, a network or an `int`, keeping any bits past the length. In `TimestampOption`, `timestamp` is `None` past the year 9999, `pmip_timestamp` holding the value, where 1.5.x raised `ValueError` for a time past it, or late enough in its last second to round up to the year 10000. It is read without summing the second and its fraction as a `float`, which was off by up to 15 microseconds past `2**36` seconds. - **Changed** -- **a breaking change to** construction: a checksum left out of `make()` is computed rather than zero, so the octets built change ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). `IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and `OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning `PCAPKIT_CHECKSUM_COMPUTE` off restores zero. @@ -58,10 +58,14 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** building `PCAPNG` blocks: a field given a value it cannot hold raises `ProtocolError`, where 1.5.x wrote it masked ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2671](https://github.com/JarryShaw/PyPCAPKit/pull/2671)). The Section Header Block's Section Length takes `-2**63` to `2**64 - 1`, `-1` and `0xFFFFFFFFFFFFFFFF` writing the same octets, and refuses anything else, where `2**64` wrote `0`, `True` wrote `1` and `None` the `-1` sentinel. The obsolete Packet Block's Drops Count takes `0` to `0xFFFF`, where `-1` wrote `0xFFFF`, `0x10000` wrote `0` and `True` wrote `1`. The `if_tzone` and `if_tsoffset` options refuse a value outside their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads `PCAP-NG: invalid [IDB] snaplen: 4294967296`, where it read `PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296`. +- **Changed** -- **a breaking change to** construction: a `make()` field holding an enumeration takes the member or an `int` code, resolved with the enumeration's `get`, and the `X_default`, `X_namespace` and `X_reversed` keywords beside such a field are removed with no alias ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). The makers of IPv4, IPv6 and its extension headers, IPX, TCP, ARP, Ethernet, VLAN, L2TPv2, OSPF, HTTP, the PCAP header and PCAP-NG took them in 1.5.x, and construction given one raises `UnsupportedCall`, as `IPv4: unexpected keyword(s): 'protocol_default' (did you mean 'protocol'?)`. A name raises `ProtocolError`, as `IPv4: invalid protocol: 'TCP'`, where 1.5.x resolved it against the field's `X_namespace` or an enumeration it inferred from the caller's frame. So does a name given as NGAP's `procedure` or `criticality`, which 1.5.x resolved to its member. A caller holding a name resolves it first with `Protocol.resolve_index`, given the namespace, a default and whether a `dict` namespace maps names to codes. A code is checked against the field's width as an integer field's is, above, so `IPv4(protocol=256)` raises `IPv4: invalid protocol: 256`, where it wrote `0x00`, and `IPv4(protocol=True)` raises, where it wrote `0x01`. So is the code of an option, sub-option or chunk given as a `(code, arguments)` pair: MH's option type 300 raises `MH: invalid option type: 300` and TCP's option kind 300 `TCP: invalid option kind: 300`, where 1.5.x wrote `0x2C` ([#2693](https://github.com/JarryShaw/PyPCAPKit/issues/2693)). An `int` HTTP/1 status given without a `message` writes its registry phrase, as a member did, so `404` writes `404 Not Found`, `299` `299 Unassigned` and `600` `600 Unknown`, where 1.5.x wrote the code with an empty phrase. + - **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. - **Changed** -- **a breaking change to** `NoPayload`, which moves from `pcapkit.protocols.misc.null` to `pcapkit.protocols.misc.no_payload`, and its data model and header schema from `pcapkit.protocols.data.misc.null` and `pcapkit.protocols.schema.misc.null` to `pcapkit.protocols.data.misc.no_payload` and `pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old paths, as a protocol class takes its module's name ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2636](https://github.com/JarryShaw/PyPCAPKit/pull/2636)). The class keeps its name, and `pcapkit.NoPayload` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails to load, as the pickle names the old module, though its pickled `info` loads. +- **Changed** -- **a breaking change to** the schema helpers that code outside the schema layer calls, which move to `pcapkit.corekit.schema`, a package with a module per group, with no re-export at their old paths ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). `parse_ip_address()` moves from `pcapkit.corekit.fields.ipaddress` to `pcapkit.corekit.schema.ipaddress`; `packet_byteorder()` and `byteorder_callback()` from `pcapkit.corekit.fields.byteorder`, which is removed with no alias, to `pcapkit.corekit.schema.byteorder`; `jumbo_payload_length()` from `pcapkit.protocols.schema.internet.ipv6` to `pcapkit.corekit.schema.jumbogram`; and `keep_short_read()` and `replay_short_read()` from `pcapkit.protocols.schema.schema` to `pcapkit.corekit.schema.shortread`. The header schema framework stays `pcapkit.protocols.schema`. + - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). Building it from fields takes `version`, `1` or `2`, as the other versioned bases above do, and raises `ProtocolError` without it, where 1.5.x built an HTTP/1.1 message; its `from_data` takes the member from the data model's class and refuses a `dict`, where 1.5.x took one ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`, which `HTTPv1` refuses, where 1.5.x wrote the HTTP/2 preface ahead of the HTTP/1 message ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). - **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. @@ -78,11 +82,11 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). The GraalPy leg there splits the long shards whose xdist workers outgrew their heap and died, each sized from its tests' JUnit seconds to run in under 5 minutes ([#2713](https://github.com/JarryShaw/PyPCAPKit/issues/2713), [#2720](https://github.com/JarryShaw/PyPCAPKit/pull/2720)). - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). @@ -104,13 +108,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). -- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside zero-filled to its length, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Each item filled is a schema object rather than a zero octet, so `ZERO_PAD_BUDGET_RATIO` bounds the fill from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. +- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside zero-filled to its length, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Each item filled is a schema object rather than a zero octet, so `ZERO_PAD_BUDGET_RATIO` bounds the fill from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. If the items then fail, the fill stays charged, as a field's does, and the octets it counted as read are given back ([#2705](https://github.com/JarryShaw/PyPCAPKit/issues/2705), [#2706](https://github.com/JarryShaw/PyPCAPKit/pull/2706)). A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. - **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses, with the octets its settings lack read as zeros, as a cut `DATA` frame's data is, and rebuilds as captured ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). -- **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). +- **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). Its header schema cut inside the padding of its last chunk, parameter or cause, or such an item's own schema cut inside its padding, packs back as captured, where 1.5.x packed it at full width, 34 octets in and 36 out; and a header schema cut inside an option area packs back the octets read, where a TCP header schema of data offset 8 cut 25 octets in gave 32 octets from `bytes()` and 23 from `pack()` ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2685](https://github.com/JarryShaw/PyPCAPKit/issues/2685), [#2682](https://github.com/JarryShaw/PyPCAPKit/pull/2682)). -- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). +- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). diff --git a/docs/source/changelog/1.5.0.rst b/docs/source/changelog/1.5.0.rst index ac2196687c..31566784be 100644 --- a/docs/source/changelog/1.5.0.rst +++ b/docs/source/changelog/1.5.0.rst @@ -414,9 +414,9 @@ Added subclasses that are also a :class:`.MultiDict` and an :class:`.OrderedMultiDict`, so a repeated key keeps every value. The data models' option lists use them (:issue:`1484`, :pr:`1523`). -* :mod:`pcapkit.corekit.fields.byteorder`, holding ``packet_byteorder`` and - ``byteorder_callback``, the byte-order helpers the PCAP and PCAP-NG schemas share - (:pr:`1528`, :issue:`1711`, :pr:`1715`). +* :mod:`pcapkit.corekit.fields.byteorder `, holding + ``packet_byteorder`` and ``byteorder_callback``, the byte-order helpers the PCAP and + PCAP-NG schemas share (:pr:`1528`, :issue:`1711`, :pr:`1715`). * :func:`~pcapkit.corekit.io.can_seek`, whether a stream really seeks: on Windows a pipe answers ``seekable()`` with ``True`` while its ``seek`` does nothing. It trusts ``seekable()`` only for a regular file or a stream with no descriptor. diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 6ee171527e..938174d133 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -44,7 +44,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and :func:`~pcapkit.foundation.registry.protocols.link.register_frad` registers a - dissector for a further NLPID (:pr:`2527`). + dissector for a further NLPID (:pr:`2527`). A Q.922 address other than the + 2-octet form, its EA bits other than 0 then 1, is refused with + :exc:`~pcapkit.utilities.exceptions.ProtocolError` unless the capture cut it + short, and is kept as ``Raw`` with nothing logged behind ``LinuxSLL`` or a + ``LINKTYPE_FRELAY`` capture. A header the capture cut short, before its NLPID + included, rebuilds as captured and logs nothing (:issue:`2710`, :pr:`2719`). * **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC @@ -250,7 +255,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. registers a parser and constructor for a further type (:issue:`2514`, :pr:`2595`). ``make()`` refuses a value too wide for its field, in the record header or in the content it builds, with - :exc:`~pcapkit.utilities.exceptions.StructError` (:issue:`2614`, :pr:`2638`). + :exc:`~pcapkit.utilities.exceptions.StructError` (:issue:`2614`, :pr:`2638`), + and a name, or a code too wide, for an enumeration field with + :exc:`~pcapkit.utilities.exceptions.ProtocolError`, as the other makers do, + below (:issue:`2594`, :pr:`2692`). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is @@ -386,7 +394,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. of 0, as TCP segmentation offload leaves it, and behind an AH there, so one the capture cut short is ``TRUNCATED`` with no warning; a GRE or IGMP layer read directly from octets that end inside its header or message is - ``TRUNCATED`` too (:issue:`2689`, :pr:`2699`). + ``TRUNCATED`` too (:issue:`2689`, :pr:`2699`). A layer ``from_data`` rebuilds + from a cut capture, from its data model or its :class:`dict`, verifies as its + parse did, so a cut GRE, ICMP, IGMP or OSPF layer is ``TRUNCATED`` with no + warning (:issue:`2703`, :pr:`2712`). * **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, @@ -401,12 +412,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. classmethods the IPX and Loopback wrappers call (:issue:`2511`, :pr:`2521`). -* **Added** -- :func:`pcapkit.corekit.fields.strings.pad_length`, the octets of +* **Added** -- :func:`pcapkit.corekit.schema.padding.pad_length`, the octets of padding that take a length to an ``align``-octet boundary, and a :class:`~pcapkit.corekit.fields.strings.PaddingField` sized by it, given ``align`` and ``after`` -- the boundary, and the field or callable whose length the padding follows -- in place of ``length`` (:issue:`2614`, - :pr:`2624`). + :pr:`2624`), in the :mod:`pcapkit.corekit.schema` package of helpers shared + across the schema layer, given below (:issue:`2674`, :pr:`2704`). * **Added** -- :mod:`pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- ``beholding``, @@ -467,12 +479,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. :exc:`~pcapkit.utilities.exceptions.FieldValueError` rebuilding it; inside IPv6 the header is ``Raw`` and the packet rebuilds as captured. -* **Changed** -- **a breaking change to** - :class:`~pcapkit.protocols.data.internet.mh.QualityOfServiceOption`, an MH - Quality-of-Service option's data model, which gains a required - ``tc_reserved`` after ``dscp``: the two reserved bits of the traffic class, - which 1.5.x dropped, so a traffic class of ``0xB9`` rebuilt as ``0xB8`` - (:issue:`2684`, :pr:`2700`). Building the model without it raises +* **Changed** -- **a breaking change to** MH option data models. + :class:`~pcapkit.protocols.data.internet.mh.QualityOfServiceOption`, the + Quality-of-Service option's, gains a required ``tc_reserved`` after + ``dscp``: the two reserved bits of the traffic class, which 1.5.x dropped, + so a traffic class of ``0xB9`` rebuilt as ``0xB8`` (:issue:`2684`, + :pr:`2700`). Building the model without it raises :exc:`TypeError`, and :class:`~pcapkit.protocols.internet.mh.MH`'s ``make()`` takes it, ``0`` by default. The MN-ID and Service Selection makers count the octets an identifier packs as, so an NAI of ``'usér@x'`` @@ -482,6 +494,22 @@ HTTP/1 and HTTP/2 classes under the RFC names. CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses through the long form of its DER length, where 1.5.x misread the length and raised :exc:`~pcapkit.utilities.exceptions.ProtocolError`. + :class:`~pcapkit.protocols.data.internet.mh.MobileNetworkPrefixOption`, the + Mobile Network Prefix option's, gains a required ``prefix_length``, and its + ``prefix`` is the :class:`~ipaddress.IPv6Address` captured, where it was an + :class:`~ipaddress.IPv6Network`, so a prefix with bits set past its length, + or a length past 128, parses and rebuilds, where 1.5.x raised a bare + :exc:`ValueError` such as ``2001:db8::1/64 has host bits set`` + (:issue:`2702`, :pr:`2709`). Its maker takes ``prefix_length``, from 0 to + 255, where 1.5.x ignored it and wrote the length ``prefix`` gave, 128 for an + address; and it takes ``prefix`` as an address, a network or an + :class:`int`, keeping any bits past the length. In + :class:`~pcapkit.protocols.data.internet.mh.TimestampOption`, ``timestamp`` + is :data:`None` past the year 9999, ``pmip_timestamp`` holding the value, + where 1.5.x raised :exc:`ValueError` for a time past it, or late enough in + its last second to round up to the year 10000. It is read without summing + the second and its fraction as a :class:`float`, which was off by up to 15 + microseconds past ``2**36`` seconds. * **Changed** -- **a breaking change to** construction: a checksum left out of ``make()`` is computed rather than zero, so the octets built change @@ -582,6 +610,35 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``PCAP-NG: invalid [IDB] snaplen: 4294967296``, where it read ``PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296``. +* **Changed** -- **a breaking change to** construction: a ``make()`` field + holding an enumeration takes the member or an :class:`int` code, resolved + with the enumeration's ``get``, and the ``X_default``, ``X_namespace`` and + ``X_reversed`` keywords beside such a field are removed with no alias + (:issue:`2594`, :pr:`2692`). The makers of IPv4, IPv6 and its extension + headers, IPX, TCP, ARP, Ethernet, VLAN, L2TPv2, OSPF, HTTP, the PCAP header + and PCAP-NG took them in 1.5.x, and construction given one raises + :exc:`~pcapkit.utilities.exceptions.UnsupportedCall`, as + ``IPv4: unexpected keyword(s): 'protocol_default' (did you mean 'protocol'?)``. + A name raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, as + ``IPv4: invalid protocol: 'TCP'``, where 1.5.x resolved it against the + field's ``X_namespace`` or an enumeration it inferred from the caller's + frame. So does a name given as NGAP's ``procedure`` or ``criticality``, + which 1.5.x resolved to its member. A caller holding a name resolves it + first with + :meth:`Protocol.resolve_index `, + given the namespace, a default and whether a :class:`dict` namespace maps + names to codes. A code is checked against the field's width as an integer + field's is, above, so ``IPv4(protocol=256)`` raises + ``IPv4: invalid protocol: 256``, where it wrote ``0x00``, and + ``IPv4(protocol=True)`` raises, where it wrote ``0x01``. So is the code of + an option, sub-option or chunk given as a ``(code, arguments)`` pair: MH's + option type 300 raises ``MH: invalid option type: 300`` and TCP's option + kind 300 ``TCP: invalid option kind: 300``, where 1.5.x wrote ``0x2C`` + (:issue:`2693`). An :class:`int` HTTP/1 status given without a ``message`` + writes its registry phrase, as a member did, so ``404`` writes + ``404 Not Found``, ``299`` ``299 Unassigned`` and ``600`` ``600 Unknown``, + where 1.5.x wrote the code with an empty phrase. + * **Changed** -- **a breaking change to** :class:`~pcapkit.protocols.application.drarp.DRARP` and :class:`~pcapkit.protocols.link.inarp.InARP`, which move from @@ -606,6 +663,26 @@ HTTP/1 and HTTP/2 classes under the RFC names. to load, as the pickle names the old module, though its pickled ``info`` loads. +* **Changed** -- **a breaking change to** the schema helpers that code outside + the schema layer calls, which move to :mod:`pcapkit.corekit.schema`, a + package with a module per group, with no re-export at their old paths + (:issue:`2674`, :pr:`2704`). + :func:`~pcapkit.corekit.schema.ipaddress.parse_ip_address` moves from + ``pcapkit.corekit.fields.ipaddress`` to + :mod:`pcapkit.corekit.schema.ipaddress`; + :func:`~pcapkit.corekit.schema.byteorder.packet_byteorder` and + :func:`~pcapkit.corekit.schema.byteorder.byteorder_callback` from + ``pcapkit.corekit.fields.byteorder``, which is removed with no alias, to + :mod:`pcapkit.corekit.schema.byteorder`; + :func:`~pcapkit.corekit.schema.jumbogram.jumbo_payload_length` from + ``pcapkit.protocols.schema.internet.ipv6`` to + :mod:`pcapkit.corekit.schema.jumbogram`; and + :func:`~pcapkit.corekit.schema.shortread.keep_short_read` and + :func:`~pcapkit.corekit.schema.shortread.replay_short_read` from + ``pcapkit.protocols.schema.schema`` to + :mod:`pcapkit.corekit.schema.shortread`. The header schema framework stays + :mod:`pcapkit.protocols.schema`. + * **Changed** -- **a breaking change to** HTTP: parsing octets through :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields and its ``from_data`` give the @@ -765,7 +842,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. layout read under it is a class-level registry map, with its ``register_`` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never - built inside the map (:issue:`2576`, :pr:`2657`). + built inside the map (:issue:`2576`, :pr:`2657`). A ``make()`` field holding + an enumeration takes the member or an :class:`int` code, never a name, + checked against the field's width and resolved with the ``get`` of an + enumeration named at the call, never one read from the caller's frame, and + no keyword group sits beside it (:issue:`2594`, :pr:`2692`). A helper that code outside the schema layer + calls lives in :mod:`pcapkit.corekit.schema`, a module per group, as does a + helper in one protocol's schema module once code outside that protocol uses + it and it is generic enough to reuse, with no re-export at its old path + (:issue:`2674`, :pr:`2704`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -788,7 +873,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. workflow's Windows leg passes pytest the modules it ignores as an ``@``-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started - (:issue:`2678`, :pr:`2681`). + (:issue:`2678`, :pr:`2681`). The GraalPy leg there splits the long shards + whose xdist workers outgrew their heap and died, each sized from its tests' + JUnit seconds to run in under 5 minutes (:issue:`2713`, :pr:`2720`). * **Changed** -- once a release's milestone is closed while ``__version__`` still names that release, the milestone workflow gives an item the lowest @@ -873,7 +960,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. bounds the fill from its first octet, with neither the free band of :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than - :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_LENGTH` is refused. A TCP + :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_LENGTH` is refused. If the + items then fail, the fill stays charged, as a field's does, and the octets + it counted as read are given back (:issue:`2705`, :pr:`2706`). A TCP ``SACK`` option running past the data raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, ``Field options has an option that runs past the end of the data: ...``, @@ -895,7 +984,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. an IPv6 Payload Length or a caller's ``length``, rebuilt through ``from_data`` without the octets after its last whole chunk, which its header schema's ``bytes()`` counted twice. It rebuilds as captured, as one - parsed with no length declared did (:issue:`2673`, :pr:`2679`). + parsed with no length declared did (:issue:`2673`, :pr:`2679`). Its header + schema cut inside the padding of its last chunk, parameter or cause, or such + an item's own schema cut inside its padding, packs back as captured, where + 1.5.x packed it at full width, 34 octets in and 36 out; and a header schema + cut inside an option area packs back the octets read, where a TCP header + schema of data offset 8 cut 25 octets in gave 32 octets from ``bytes()`` and + 23 from ``pack()`` (:issue:`2680`, :issue:`2685`, :pr:`2682`). * **Fixed** -- a :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` Decryption Secrets Block that ``make()`` built from a parsed ZigBee NWK or APS key @@ -905,7 +1000,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. where 28 were captured. It is written as captured, and ``pcapkit.protocols.schema.misc.pcapng.zigbee_key_held`` returns a schema whose :func:`bytes` and :func:`len` are those of the octets held, where they - were the fixed width's (:issue:`2688`, :pr:`2697`). + were the fixed width's (:issue:`2688`, :pr:`2697`). Rebuilt by ``from_data`` + from the block's :class:`dict`, a TLS or WireGuard key log was written from its + entries under a ``# generated by PyPCAPKit`` line, dropping its comments and + the lines its entries do not hold, and a ZigBee key cut short of its fixed + width was written at that width, zero-filled. The TLS, WireGuard and ZigBee + secrets makers take the captured text or octets as ``data``, so the block + rebuilds as captured from its :class:`dict`, as from its data model + (:issue:`2698`, :pr:`2707`). * **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or socket crawler, without ``requests`` installed lost an entry of From 41f7e7e8efaa67b1033bf2ccf4b1d6767437c390 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Mon, 2 Nov 2026 23:52:20 -0500 Subject: [PATCH 20/27] docs(changelog): fold the makers' fixed-size octets, pcapkit.utilities.pickle, FRAD's pad and the TCP and HIP maker fixes into 1.6.0 (#2449) - Breaking: a fixed-size octet field given any other width raises, where 1.5.x padded or cut it, and IPv4's Timestamp option refuses a slot count past what it holds, where 1.5.x cut it (#2691); folded into the IPv4 to HIP makers' range-check entry, with the ZigBee keys into the PCAP-NG entry. Old values measured against v1.5.1. - Added: pcapkit.utilities.pickle, whose ReduceAsProtocol2 lets Info, Schema and the reassembly engines pickle under protocols 0 and 1 on GraalPy, and numbers._reduce_unregistered_member's removal (#2722). - Fixed: TCP's raw-octet options, HIP's default version, HIP's ENCRYPTED IV and its cipher on parse (#2724). - Folded into unreleased entries: FRAD's pad field and make(pad=) (#2721). - No entry for the docstring-only 596dcfaf8, the test-only 149cced7f, or #2724's MH annotation. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 10 ++++-- docs/source/changelog/1.6.0.rst | 54 +++++++++++++++++++++++++++++++-- 2 files changed, 59 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c065e9b13a..1eeca89d7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `LinuxSLL1` and `LinuxSLL2`, dissectors for the Linux "cooked" capture headers `tcpdump -i any` writes, under `LINKTYPE_LINUX_SLL` (113) and `LINKTYPE_LINUX_SLL2` (276), on a `LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them ([#2438](https://github.com/JarryShaw/PyPCAPKit/issues/2438), [#2443](https://github.com/JarryShaw/PyPCAPKit/pull/2443)). `'LinuxSLL'` selects either version, and each version's own name selects that version alone. `protocol_type` dispatches onward as an EtherType, except under the `arphrd_type` values that carry none -- `FRAD`, `NETLINK`, `IPGRE`/`IP6GRE` and `IEEE80211_RADIOTAP` -- each of which has a dissector of its own, below ([#2460](https://github.com/JarryShaw/PyPCAPKit/issues/2460)), reached through `LinuxSLL`'s own registry keyed by `arphrd_type`, which `register_arphrd()` extends ([#2510](https://github.com/JarryShaw/PyPCAPKit/issues/2510), [#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)), as does a protocol class declaring `code=ARPHRD.X`, since `ARPHRD` names `LinuxSLL` as its `__proto_dest__` ([#2533](https://github.com/JarryShaw/PyPCAPKit/issues/2533), [#2598](https://github.com/JarryShaw/PyPCAPKit/pull/2598)). A parse told to stop there -- at `LinuxSLL`, the version itself or the link layer -- keeps the payload `Raw` on either path ([#2532](https://github.com/JarryShaw/PyPCAPKit/issues/2532), [#2587](https://github.com/JarryShaw/PyPCAPKit/pull/2587)). The hardware types are `ARPHRD`, generated by `pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It names only the `` values that table lists, so any other, `ARPHRD_ETHER` included, reads as `Unassigned` ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2497](https://github.com/JarryShaw/PyPCAPKit/pull/2497)), and `pcapkit.const` exports it as `Linux_ARPHRD` ([#2661](https://github.com/JarryShaw/PyPCAPKit/issues/2661), [#2663](https://github.com/JarryShaw/PyPCAPKit/pull/2663)). -- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)), as do PCAP and PCAP-NG frames of `LINKTYPE_FRELAY` (107) ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). A Q.922 address other than the 2-octet form, its EA bits other than 0 then 1, is refused with `ProtocolError` unless the capture cut it short, and is kept as `Raw` with nothing logged behind `LinuxSLL` or a `LINKTYPE_FRELAY` capture. A header the capture cut short, before its NLPID included, rebuilds as captured and logs nothing ([#2710](https://github.com/JarryShaw/PyPCAPKit/issues/2710), [#2719](https://github.com/JarryShaw/PyPCAPKit/pull/2719)). +- **Added** -- `FRAD`, a dissector for Frame Relay's multiprotocol encapsulation ([RFC 2427](https://datatracker.ietf.org/doc/html/rfc2427), obsoleting [RFC 1490](https://datatracker.ietf.org/doc/html/rfc1490)), which `LinuxSLL` dispatches to for `ARPHRD_FRAD` ([#2499](https://github.com/JarryShaw/PyPCAPKit/pull/2499)), as do PCAP and PCAP-NG frames of `LINKTYPE_FRELAY` (107) ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or SNAP OUI is kept as `Raw`, and `register_frad()` registers a dissector for a further NLPID ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). A Q.922 address other than the 2-octet form, its EA bits other than 0 then 1, is refused with `ProtocolError` unless the capture cut it short, and is kept as `Raw` with nothing logged behind `LinuxSLL` or a `LINKTYPE_FRELAY` capture. A header the capture cut short, before its NLPID included, rebuilds as captured and logs nothing ([#2710](https://github.com/JarryShaw/PyPCAPKit/issues/2710), [#2719](https://github.com/JarryShaw/PyPCAPKit/pull/2719)). The data model records as `pad` whether the one `0x00` pad octet [RFC 2427 Section 3](https://datatracker.ietf.org/doc/html/rfc2427#section-3) allows ahead of an NLPID was captured, so a frame rebuilds with its pad or without it, as captured; `make()` takes `pad`, by default writing one ahead of a SNAP header or an NLPID of 0, and refuses `pad=False` with an NLPID of 0 ([#2716](https://github.com/JarryShaw/PyPCAPKit/issues/2716), [#2721](https://github.com/JarryShaw/PyPCAPKit/pull/2721)). - **Added** -- `Netlink`, a dissector for the fixed 16-octet `nlmsghdr` framing of Linux's kernel/userspace IPC transport (the kernel's `netlink(7)` man page; [RFC 3549 Section 2.3.2](https://datatracker.ietf.org/doc/html/rfc3549#section-2.3.2)), which `LinuxSLL` dispatches to for `ARPHRD_NETLINK` ([#2500](https://github.com/JarryShaw/PyPCAPKit/issues/2500), [#2507](https://github.com/JarryShaw/PyPCAPKit/pull/2507)). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; `nlmsg_byteorder()` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message in the same capture, is kept as `Raw` unless `register_netlink()` registers a dissector for its `nlmsg_type` ([#2527](https://github.com/JarryShaw/PyPCAPKit/pull/2527)). @@ -42,6 +42,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `pcapkit.utilities.contextvars`, holding the context variables more than one module sets or reads -- `beholding`, `frame_depth`, `frame_length` and `rebuilding` -- and importing nothing from `pcapkit` ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). `frame_length` holds the captured and original length of the frame being dissected, as its PCAP record or PCAP-NG packet block states them, and is `None` outside a frame and inside an IP layer's payload ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)), unless the layer states no length for it: an IPv4 Total Length of 0, or an AH that IPv4 carries, hands the frame's lengths on ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). +- **Added** -- `pcapkit.utilities.pickle`, holding the `pickle` reductions classes share ([#2714](https://github.com/JarryShaw/PyPCAPKit/issues/2714), [#2722](https://github.com/JarryShaw/PyPCAPKit/pull/2722)): `ReduceAsProtocol2`, which reduces under every protocol as protocol 2 does; `ReduceToName`, which reduces a singleton to the name holding it; `ReduceToItems`, which reduces a multi-valued mapping to a call of its class on its item pairs; and `reduce_to()`, which makes one instance reduce to a given call. `Info`, `Schema` and the reassembly engines take `ReduceAsProtocol2`, so on GraalPy they pickle under protocols 0 and 1, where 1.5.x raised `TypeError: typing.Generic() takes no arguments`. `pcapkit.corekit.fields.numbers._reduce_unregistered_member` is removed with no alias, `reduce_to` installing the reduction it built. + - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. - **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. A mobility option whose fields do not take its Length is refused too, and so are such a sub-option, QoS attribute or CGA extension inside one, a target care-of address neither IPv4 nor IPv6, and CGA Parameters whose fixed fields or public key run past the option, where 1.5.x read octets that were not the option's and rebuilt another ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). So an MN-ID option of Length 0 raises `MH: [OptNo 8] invalid option length: 0`, where 1.5.x read its fields from the octets after it and rebuilt it with Length 17, and a Flow Identification option's 8-octet target care-of address raises `MH: [FID SubOpt 5] invalid sub-option length: 10`, where 1.5.x parsed it and raised `FieldValueError` rebuilding it; inside IPv6 the header is `Raw` and the packet rebuilds as captured. @@ -54,9 +56,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** building `TCP`, `UDP` and `SCTP`: a header or option field given an integer it cannot hold raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and a fixed-size octet field given the wrong number of octets raises, where it was padded or cut ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). So `TCP(window=70000)` raises `TCP: invalid window: 70000`, where it wrote `0x1170`; `TCP(seq_no=-1)` and `SCTP(vtag=-1)` raise, where they wrote `0xFFFFFFFF`; a UDP payload of `0xFFF8` octets raises, where its Length was written `0`; and `TCP(checksum=b'\x01')` raises, where it wrote `0100`. The TCP User Timeout refuses a negative, `True` or non-numeric timeout by name, where a negative one raised a bare `ValueError` and `True` built one second. The Quick-Start TTL Diff, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) defines modulo 256, wraps as in 1.5.x. -- **Changed** -- **a breaking change to** building `IPv4`, `HOPOPT`, `IPv6_Opts`, `IPv6_Route`, `MH` and `HIP`, as TCP's, UDP's and SCTP's above: a header, option or parameter field given an integer it cannot hold, or `True` where it is wider than one bit, raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and so does a header too long for its length field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). So `IPv4(ttl=256)` raises `IPv4: invalid TTL: 256`, where it wrote `0`, and `IPv4(id=True)` raises, where it wrote `1`; a Binding Update's sequence number of `-1` raises `MH: invalid sequence number: -1`, where it wrote `0xFFFF`; and a HOPOPT or HIP header too long for its length field raises `HOPOPT: invalid Hdr Ext Len: 256` or `HIPv2: invalid header length: 256`, where that field was written `0`. A bit field raised `FieldValueError` for a value too wide and a bare `ValueError` or `OverflowError` for a negative one, where IPv4 options too long for the IHL, say, raise `IPv4: invalid IHL: 16`. A PDM scale past 255, given or needed for its delta, raises `HOPOPT: invalid PDM scale of delta time last received: 256` say, where it warned with `ProtocolWarning` and was written wrapped. HIP's lifetimes refuse `True`, which built as `1`, and a NaN or infinite one, which raised a bare `ValueError` or `OverflowError`. An MH prefix length is held to its field's width alone, as a parse keeps it -- 0 to 63 in the IPv4 Address Acknowledgement and the IPv4 Home Address Request and Reply, 0 to 255 in the Delegated Mobile Network Prefix -- so a 33 builds and a parsed 63 rebuilds, where 1.5.x refused one past the address, as `MH: [OptNo 30] invalid prefix length: 33` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Where one of these checks replaces a refusal 1.5.x made, its message drops the option's or parameter's tag, as `HOPOPT: invalid RPL reserved flags: 32`, where it read `HOPOPT: [OptNo 99] invalid RPL reserved flags: 32`. `MH._decode_signed`, which decoded the geo-location degrees, is removed with no alias; the base class's `_read_sint` decodes them. +- **Changed** -- **a breaking change to** building `IPv4`, `HOPOPT`, `IPv6_Opts`, `IPv6_Route`, `MH` and `HIP`, as TCP's, UDP's and SCTP's above: a header, option or parameter field given an integer it cannot hold, or `True` where it is wider than one bit, raises `ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and so does a header too long for its length field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2677](https://github.com/JarryShaw/PyPCAPKit/pull/2677)). So `IPv4(ttl=256)` raises `IPv4: invalid TTL: 256`, where it wrote `0`, and `IPv4(id=True)` raises, where it wrote `1`; a Binding Update's sequence number of `-1` raises `MH: invalid sequence number: -1`, where it wrote `0xFFFF`; and a HOPOPT or HIP header too long for its length field raises `HOPOPT: invalid Hdr Ext Len: 256` or `HIPv2: invalid header length: 256`, where that field was written `0`. A bit field raised `FieldValueError` for a value too wide and a bare `ValueError` or `OverflowError` for a negative one, where IPv4 options too long for the IHL, say, raise `IPv4: invalid IHL: 16`. A PDM scale past 255, given or needed for its delta, raises `HOPOPT: invalid PDM scale of delta time last received: 256` say, where it warned with `ProtocolWarning` and was written wrapped. HIP's lifetimes refuse `True`, which built as `1`, and a NaN or infinite one, which raised a bare `ValueError` or `OverflowError`. An MH prefix length is held to its field's width alone, as a parse keeps it -- 0 to 63 in the IPv4 Address Acknowledgement and the IPv4 Home Address Request and Reply, 0 to 255 in the Delegated Mobile Network Prefix -- so a 33 builds and a parsed 63 rebuilds, where 1.5.x refused one past the address, as `MH: [OptNo 30] invalid prefix length: 33` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Where one of these checks replaces a refusal 1.5.x made, its message drops the option's or parameter's tag, as `HOPOPT: invalid RPL reserved flags: 32`, where it read `HOPOPT: [OptNo 99] invalid RPL reserved flags: 32`. `MH._decode_signed`, which decoded the geo-location degrees, is removed with no alias; the base class's `_read_sint` decodes them. A fixed-size octet field given octets of any other width raises as well, where 1.5.x padded or cut them, so `IPv4(checksum=b'\x01')` raises `IPv4: invalid checksum: must be 2 octets, not b'\x01'`, where it wrote `0100`, and a 300-octet CALIPSO checksum raises, where its first two octets were written ([#2676](https://github.com/JarryShaw/PyPCAPKit/issues/2676), [#2691](https://github.com/JarryShaw/PyPCAPKit/pull/2691)). The fields are the IPv4, MH, HIP, IPX and OSPF checksums, the CALIPSO checksum, MH cookies and keygen tokens, HIP opaque and payload data, IPX addresses, and the reserved and unused octets of the IPv6 extension headers, MH, HIP and OSPF; ICMP's, IGMP's and IP_Pseudo's makers, added above, check theirs the same way. A model parsed from a capture, one cut inside such a field included, rebuilds as captured. IPv4's Timestamp option refuses a slot count past what its 40 octets hold, nine timestamps or four address and timestamp pairs, as `IPv4: invalid timestamp count: 10`, where 1.5.x cut it to fit. -- **Changed** -- **a breaking change to** building `PCAPNG` blocks: a field given a value it cannot hold raises `ProtocolError`, where 1.5.x wrote it masked ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2671](https://github.com/JarryShaw/PyPCAPKit/pull/2671)). The Section Header Block's Section Length takes `-2**63` to `2**64 - 1`, `-1` and `0xFFFFFFFFFFFFFFFF` writing the same octets, and refuses anything else, where `2**64` wrote `0`, `True` wrote `1` and `None` the `-1` sentinel. The obsolete Packet Block's Drops Count takes `0` to `0xFFFF`, where `-1` wrote `0xFFFF`, `0x10000` wrote `0` and `True` wrote `1`. The `if_tzone` and `if_tsoffset` options refuse a value outside their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads `PCAP-NG: invalid [IDB] snaplen: 4294967296`, where it read `PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296`. +- **Changed** -- **a breaking change to** building `PCAPNG` blocks: a field given a value it cannot hold raises `ProtocolError`, where 1.5.x wrote it masked ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2671](https://github.com/JarryShaw/PyPCAPKit/pull/2671)). The Section Header Block's Section Length takes `-2**63` to `2**64 - 1`, `-1` and `0xFFFFFFFFFFFFFFFF` writing the same octets, and refuses anything else, where `2**64` wrote `0`, `True` wrote `1` and `None` the `-1` sentinel. The obsolete Packet Block's Drops Count takes `0` to `0xFFFF`, where `-1` wrote `0xFFFF`, `0x10000` wrote `0` and `True` wrote `1`. The `if_tzone` and `if_tsoffset` options refuse a value outside their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads `PCAP-NG: invalid [IDB] snaplen: 4294967296`, where it read `PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296`. A ZigBee NWK or APS key of other than 16 octets raises `PCAP-NG: invalid [ZigBee NWK Key] key: must be 16 octets, not ...` say, where 1.5.x zero-filled or cut it ([#2676](https://github.com/JarryShaw/PyPCAPKit/issues/2676), [#2691](https://github.com/JarryShaw/PyPCAPKit/pull/2691)). - **Changed** -- **a breaking change to** construction: a `make()` field holding an enumeration takes the member or an `int` code, resolved with the enumeration's `get`, and the `X_default`, `X_namespace` and `X_reversed` keywords beside such a field are removed with no alias ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). The makers of IPv4, IPv6 and its extension headers, IPX, TCP, ARP, Ethernet, VLAN, L2TPv2, OSPF, HTTP, the PCAP header and PCAP-NG took them in 1.5.x, and construction given one raises `UnsupportedCall`, as `IPv4: unexpected keyword(s): 'protocol_default' (did you mean 'protocol'?)`. A name raises `ProtocolError`, as `IPv4: invalid protocol: 'TCP'`, where 1.5.x resolved it against the field's `X_namespace` or an enumeration it inferred from the caller's frame. So does a name given as NGAP's `procedure` or `criticality`, which 1.5.x resolved to its member. A caller holding a name resolves it first with `Protocol.resolve_index`, given the namespace, a default and whether a `dict` namespace maps names to codes. A code is checked against the field's width as an integer field's is, above, so `IPv4(protocol=256)` raises `IPv4: invalid protocol: 256`, where it wrote `0x00`, and `IPv4(protocol=True)` raises, where it wrote `0x01`. So is the code of an option, sub-option or chunk given as a `(code, arguments)` pair: MH's option type 300 raises `MH: invalid option type: 300` and TCP's option kind 300 `TCP: invalid option kind: 300`, where 1.5.x wrote `0x2C` ([#2693](https://github.com/JarryShaw/PyPCAPKit/issues/2693)). An `int` HTTP/1 status given without a `message` writes its registry phrase, as a member did, so `404` writes `404 Not Found`, `299` `299 Unassigned` and `600` `600 Unknown`, where 1.5.x wrote the code with an empty phrase. @@ -114,6 +116,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). Its header schema cut inside the padding of its last chunk, parameter or cause, or such an item's own schema cut inside its padding, packs back as captured, where 1.5.x packed it at full width, 34 octets in and 36 out; and a header schema cut inside an option area packs back the octets read, where a TCP header schema of data offset 8 cut 25 octets in gave 32 octets from `bytes()` and 23 from `pack()` ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2685](https://github.com/JarryShaw/PyPCAPKit/issues/2685), [#2682](https://github.com/JarryShaw/PyPCAPKit/pull/2682)). +- **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. + - **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 938174d133..f058c6e6d9 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -50,6 +50,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. short, and is kept as ``Raw`` with nothing logged behind ``LinuxSLL`` or a ``LINKTYPE_FRELAY`` capture. A header the capture cut short, before its NLPID included, rebuilds as captured and logs nothing (:issue:`2710`, :pr:`2719`). + The data model records as ``pad`` whether the one ``0x00`` pad octet + :rfc:`2427#section-3` allows ahead of an NLPID was captured, so a frame + rebuilds with its pad or without it, as captured; ``make()`` takes ``pad``, + by default writing one ahead of a SNAP header or an NLPID of 0, and refuses + ``pad=False`` with an NLPID of 0 (:issue:`2716`, :pr:`2721`). * **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC @@ -430,6 +435,21 @@ HTTP/1 and HTTP/2 classes under the RFC names. the layer states no length for it: an IPv4 Total Length of 0, or an AH that IPv4 carries, hands the frame's lengths on (:issue:`2689`, :pr:`2699`). +* **Added** -- :mod:`pcapkit.utilities.pickle`, holding the :mod:`pickle` + reductions classes share (:issue:`2714`, :pr:`2722`): + :class:`~pcapkit.utilities.pickle.ReduceAsProtocol2`, which reduces under + every protocol as protocol 2 does; + :class:`~pcapkit.utilities.pickle.ReduceToName`, which reduces a singleton to + the name holding it; :class:`~pcapkit.utilities.pickle.ReduceToItems`, which + reduces a multi-valued mapping to a call of its class on its item pairs; and + :func:`~pcapkit.utilities.pickle.reduce_to`, which makes one instance reduce + to a given call. :class:`~pcapkit.corekit.infoclass.Info`, + :class:`~pcapkit.protocols.schema.schema.Schema` and the reassembly engines + take ``ReduceAsProtocol2``, so on GraalPy they pickle under protocols 0 and + 1, where 1.5.x raised ``TypeError: typing.Generic() takes no arguments``. + ``pcapkit.corekit.fields.numbers._reduce_unregistered_member`` is removed + with no alias, ``reduce_to`` installing the reduction it built. + * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as :class:`~pcapkit.protocols.misc.raw.Raw`, and the walk stops @@ -593,7 +613,20 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``HOPOPT: invalid RPL reserved flags: 32``, where it read ``HOPOPT: [OptNo 99] invalid RPL reserved flags: 32``. ``MH._decode_signed``, which decoded the geo-location degrees, is removed with no alias; the base - class's ``_read_sint`` decodes them. + class's ``_read_sint`` decodes them. A fixed-size octet field given octets of + any other width raises as well, where 1.5.x padded or cut them, so + ``IPv4(checksum=b'\x01')`` raises + ``IPv4: invalid checksum: must be 2 octets, not b'\x01'``, where it wrote + ``0100``, and a 300-octet CALIPSO checksum raises, where its first two + octets were written (:issue:`2676`, :pr:`2691`). The fields are the IPv4, + MH, HIP, IPX and OSPF checksums, the CALIPSO checksum, MH cookies and keygen + tokens, HIP opaque and payload data, IPX addresses, and the reserved and + unused octets of the IPv6 extension headers, MH, HIP and OSPF; ICMP's, + IGMP's and IP_Pseudo's makers, added above, check theirs the same way. + A model parsed from a capture, one cut inside such a field included, + rebuilds as captured. IPv4's Timestamp option refuses a slot count past what + its 40 octets hold, nine timestamps or four address and timestamp pairs, as + ``IPv4: invalid timestamp count: 10``, where 1.5.x cut it to fit. * **Changed** -- **a breaking change to** building :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` blocks: a field given a value @@ -608,7 +641,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. their signed 32 and 64 bits, where they wrote it masked, and the Interface Description Block's snaplen refusal reads ``PCAP-NG: invalid [IDB] snaplen: 4294967296``, where it read - ``PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296``. + ``PCAP-NG: [IDB] snaplen must be in range 0 to 0xFFFFFFFF: 4294967296``. A + ZigBee NWK or APS key of other than 16 octets raises + ``PCAP-NG: invalid [ZigBee NWK Key] key: must be 16 octets, not ...`` say, + where 1.5.x zero-filled or cut it (:issue:`2676`, :pr:`2691`). * **Changed** -- **a breaking change to** construction: a ``make()`` field holding an enumeration takes the member or an :class:`int` code, resolved @@ -992,6 +1028,20 @@ HTTP/1 and HTTP/2 classes under the RFC names. schema of data offset 8 cut 25 octets in gave 32 octets from ``bytes()`` and 23 from ``pack()`` (:issue:`2680`, :issue:`2685`, :pr:`2682`). +* **Fixed** -- :class:`~pcapkit.protocols.transport.tcp.TCP`'s maker, given an + option as raw octets, raised :exc:`~pcapkit.utilities.exceptions.EnumValueError` + for a kind its registry does not name, such as 255, and :exc:`IndexError` for + empty octets. The kind resolves as a ``(code, arguments)`` pair's does, so + 255 builds, and empty octets raise + :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2708`, + :pr:`2724`). ``HIP(next=59)`` wrote HIP version 6, taking the IP version for + it, and writes 2, ``make()``'s default from :rfc:`7401#section-5.1`. HIP's + ``ENCRYPTED`` maker refuses an IV given with a cipher that takes none, which + 1.5.x counted in the parameter's Length but did not write. A parsed + ``ENCRYPTED`` parameter takes its cipher from the ``HIP_CIPHER`` before it, + where 1.5.x warned ``HIP: [ParamNo 641] missing HIP_CIPHER parameter`` and + read an AES IV as encrypted data. + * **Fixed** -- a :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` Decryption Secrets Block that ``make()`` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 From e331c1a8dd2d654f1da59fecf76229505331c632 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Tue, 3 Nov 2026 00:45:11 -0500 Subject: [PATCH 21/27] docs(changelog): fold the makers' refusal of an empty raw item into 1.6.0 (#2449) - Fixed: an empty raw option, parameter, cause, sub-option, attribute, CGA extension or PCAP-NG option or record shorter than its code raises ProtocolError (#2728), into the TCP and HIP makers' Fixed entry beside TCP's own; the IPv4, HOPOPT, IPv6-Opts, SCTP, MH, HIP and PCAP-NG outcomes measured against v1.5.1. - Folded into unreleased entries: DCCP's empty raw option (#2728). - No entry for the docstring-only 596dcfaf8 and 6b63ee9a9, or the test-only 219fbbc85. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 4 ++-- docs/source/changelog/1.6.0.rst | 16 ++++++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1eeca89d7b..8f365f6c48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,7 +20,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. HTTP's base follows the same rules, a breaking change given below ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. -- **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. Its `make()` refuses a value a field cannot hold with `ProtocolError` naming the field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)). +- **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. Its `make()` refuses a value a field cannot hold with `ProtocolError` naming the field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)), and an option given as empty raw octets ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). - **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)), and a name, or a code too wide, for an enumeration field with `ProtocolError`, as the other makers do, below ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. `pcapkit.protocols.data` and `pcapkit.protocols.schema` export a `tls12_cid` record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). @@ -116,7 +116,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). Its header schema cut inside the padding of its last chunk, parameter or cause, or such an item's own schema cut inside its padding, packs back as captured, where 1.5.x packed it at full width, 34 octets in and 36 out; and a header schema cut inside an option area packs back the octets read, where a TCP header schema of data offset 8 cut 25 octets in gave 32 octets from `bytes()` and 23 from `pack()` ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2685](https://github.com/JarryShaw/PyPCAPKit/issues/2685), [#2682](https://github.com/JarryShaw/PyPCAPKit/pull/2682)). -- **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. +- **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). The other makers taking a raw item refuse empty octets the same way, as `IPv4: invalid option: b'' holds no type octet` ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). IPv4's, HOPOPT's and IPv6-Opts' options raised `IndexError`. SCTP's parameters and error causes, and MH's options, sub-options, QoS attributes and CGA extensions, raised a bare `AttributeError` alone -- or, for MH's access network identifier and LMA-controlled MAG sub-options, a `ProtocolError` for an option of length 0 -- and were dropped beside a whole item. HIP's parameters were dropped. PCAP-NG's options and Name Resolution records refuse octets shorter than their 2-octet code, where 1.5.x wrote empty or zero octets as an end-of-options or end-of-records marker, and misread one other octet as a code. `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. - **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index f058c6e6d9..422cf6dc76 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -238,7 +238,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. option parser. The registries are :mod:`pcapkit.const.dccp`, crawled from IANA. Its ``make()`` refuses a value a field cannot hold with :exc:`~pcapkit.utilities.exceptions.ProtocolError` naming the field - (:issue:`2614`, :pr:`2670`). + (:issue:`2614`, :pr:`2670`), and an option given as empty raw octets + (:issue:`2723`, :pr:`2728`). * **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` @@ -1034,7 +1035,18 @@ HTTP/1 and HTTP/2 classes under the RFC names. empty octets. The kind resolves as a ``(code, arguments)`` pair's does, so 255 builds, and empty octets raise :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2708`, - :pr:`2724`). ``HIP(next=59)`` wrote HIP version 6, taking the IP version for + :pr:`2724`). The other makers taking a raw item refuse empty octets the same + way, as ``IPv4: invalid option: b'' holds no type octet`` (:issue:`2723`, + :pr:`2728`). IPv4's, HOPOPT's and IPv6-Opts' options raised + :exc:`IndexError`. SCTP's parameters and error causes, and MH's options, + sub-options, QoS attributes and CGA extensions, raised a bare + :exc:`AttributeError` alone -- or, for MH's access network identifier and + LMA-controlled MAG sub-options, a ``ProtocolError`` for an option of length + 0 -- and were dropped beside a whole item. HIP's parameters were dropped. + PCAP-NG's options and Name Resolution records refuse octets shorter than + their 2-octet code, where 1.5.x wrote empty or zero octets as an + end-of-options or end-of-records marker, and misread one other octet as a + code. ``HIP(next=59)`` wrote HIP version 6, taking the IP version for it, and writes 2, ``make()``'s default from :rfc:`7401#section-5.1`. HIP's ``ENCRYPTED`` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed From 554b3b88504d83e5b58577e92173603c0c275b8c Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Tue, 3 Nov 2026 11:34:03 -0500 Subject: [PATCH 22/27] docs(changelog): fold #2732 and #2734 - Breaking: FrameRecord is an Info and TLSKeyLog.entries a TLSKeyLogEntries data model, not dicts (#2734); a new entry with 1.5.1's behaviour, and pickles checked loading both ways. - Fixed: MH's CGA Parameters maker builds a data model's raw extension and a schema without public_key_test (#2732), into the makers' Fixed entry. - Fixed: a capture cut inside or in front of an option area reads as a short read in MH, HOPOPT and IPv6-Opts, and an odd item list raises ProtocolError (#2731, ab7be2fb0, which the merge of 07ceaf7c1 also brings in); the short-read ruling into the conventions entry. - Old values measured against v1.5.1. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 +++-- docs/source/changelog/1.6.0.rst | 57 +++++++++++++++++++++++++++++---- 2 files changed, 57 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f365f6c48..65d83bc0a4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** the schema helpers that code outside the schema layer calls, which move to `pcapkit.corekit.schema`, a package with a module per group, with no re-export at their old paths ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). `parse_ip_address()` moves from `pcapkit.corekit.fields.ipaddress` to `pcapkit.corekit.schema.ipaddress`; `packet_byteorder()` and `byteorder_callback()` from `pcapkit.corekit.fields.byteorder`, which is removed with no alias, to `pcapkit.corekit.schema.byteorder`; `jumbo_payload_length()` from `pcapkit.protocols.schema.internet.ipv6` to `pcapkit.corekit.schema.jumbogram`; and `keep_short_read()` and `replay_short_read()` from `pcapkit.protocols.schema.schema` to `pcapkit.corekit.schema.shortread`. The header schema framework stays `pcapkit.protocols.schema`. +- **Changed** -- **a breaking change to** the mapping values a caller reads as a `dict`: each is an `Info` or a data model, not a `dict` ([#2726](https://github.com/JarryShaw/PyPCAPKit/issues/2726), [#2734](https://github.com/JarryShaw/PyPCAPKit/pull/2734)). `FrameRecord`, the frame a third-party engine's flow tracing reports, is an `Info` where 1.5.x subclassed `dict`, so `isinstance(record, dict)` is false, setting a key raises `TypeError`, and `to_dict()` exports the engine's mapping as an `OrderedMultiDict` without the `packet`, `frame_info` and `time_epoch` attributes, which stay attributes, as in 1.5.x. A parsed TLS key log's `TLSKeyLog.entries` is a `TLSKeyLogEntries` keyed by label, which `pcapkit.protocols.data` exports as `PCAPNG_TLSKeyLogEntries`, where it was a `dict`; a mapping given for it is stored as one, and `to_dict()` exports it as an `OrderedMultiDict`, where it gave a `dict`. Pickles of either load across 1.5.x and 1.6.0 in both directions. + - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). Building it from fields takes `version`, `1` or `2`, as the other versioned bases above do, and raises `ProtocolError` without it, where 1.5.x built an HTTP/1.1 message; its `from_data` takes the member from the data model's class and refuses a `dict`, where 1.5.x took one ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`, which `HTTPv1` refuses, where 1.5.x wrote the HTTP/2 preface ahead of the HTTP/1 message ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). - **Changed** -- **a breaking change to** IPv6 reassembly: a fragment overlapping another of its datagram, other than an exact duplicate, abandons the datagram, reported as `Completion.OVERLAP`, as [RFC 8200 Section 4.5](https://datatracker.ietf.org/doc/html/rfc8200#section-4.5) requires; 1.5.x overwrote the octets it overlapped, as IPv4 still does under [RFC 791](https://datatracker.ietf.org/doc/html/rfc791) ([#2547](https://github.com/JarryShaw/PyPCAPKit/issues/2547), [#2569](https://github.com/JarryShaw/PyPCAPKit/pull/2569)). `Buffer` gains a required `extent` field. @@ -84,7 +86,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). @@ -116,7 +118,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). Its header schema cut inside the padding of its last chunk, parameter or cause, or such an item's own schema cut inside its padding, packs back as captured, where 1.5.x packed it at full width, 34 octets in and 36 out; and a header schema cut inside an option area packs back the octets read, where a TCP header schema of data offset 8 cut 25 octets in gave 32 octets from `bytes()` and 23 from `pack()` ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2685](https://github.com/JarryShaw/PyPCAPKit/issues/2685), [#2682](https://github.com/JarryShaw/PyPCAPKit/pull/2682)). -- **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). The other makers taking a raw item refuse empty octets the same way, as `IPv4: invalid option: b'' holds no type octet` ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). IPv4's, HOPOPT's and IPv6-Opts' options raised `IndexError`. SCTP's parameters and error causes, and MH's options, sub-options, QoS attributes and CGA extensions, raised a bare `AttributeError` alone -- or, for MH's access network identifier and LMA-controlled MAG sub-options, a `ProtocolError` for an option of length 0 -- and were dropped beside a whole item. HIP's parameters were dropped. PCAP-NG's options and Name Resolution records refuse octets shorter than their 2-octet code, where 1.5.x wrote empty or zero octets as an end-of-options or end-of-records marker, and misread one other octet as a code. `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. +- **Fixed** -- a capture cut inside or in front of an option area reads as a short read, as a cut elsewhere in a header does ([#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)). So an `MH` message cut inside its options or reserved octets, and a `HOPOPT` or `IPv6_Opts` header cut short, parse alone or inside IPv6 and rebuild as captured, logging nothing, where 1.5.x raised `FieldValueError` or `ProtocolError` logged at `CRITICAL`, as `Field options has an option that consumed no data: at offset 0` for a HOPOPT header cut to 2 octets. A cut option stands only if its maker writes the captured octets back, and is refused quietly otherwise; one whose fields run past its area is refused, as in 1.5.x. HOPOPT's and IPv6-Opts' `make()` take `len`, the Hdr Ext Len a cut header declared. An MH item list its length leaves part of an item in, as a Flow Summary option of an odd length, raises `ProtocolError`, quiet inside IPv6, where 1.5.x raised `FieldValueError: Field fid has invalid length.` logged at `CRITICAL`. + +- **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). The other makers taking a raw item refuse empty octets the same way, as `IPv4: invalid option: b'' holds no type octet` ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). IPv4's, HOPOPT's and IPv6-Opts' options raised `IndexError`. SCTP's parameters and error causes, and MH's options, sub-options, QoS attributes and CGA extensions, raised a bare `AttributeError` alone -- or, for MH's access network identifier and LMA-controlled MAG sub-options, a `ProtocolError` for an option of length 0 -- and were dropped beside a whole item. HIP's parameters were dropped. PCAP-NG's options and Name Resolution records refuse octets shorter than their 2-octet code, where 1.5.x wrote empty or zero octets as an end-of-options or end-of-records marker, and misread one other octet as a code. `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. MH's CGA Parameters maker builds a raw extension in a `CGAParameter` data model as its `(type, arguments)` pair builds, where 1.5.x raised `AttributeError: 'bytes' object has no attribute 'type'`, and a `CGAParameter` schema without `public_key_test`, leaving the schema given unchanged, where 1.5.x raised `KeyError: 'public_key_test'` ([#2725](https://github.com/JarryShaw/PyPCAPKit/issues/2725), [#2732](https://github.com/JarryShaw/PyPCAPKit/pull/2732)). - **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 422cf6dc76..8979c45e61 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -720,6 +720,24 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.corekit.schema.shortread`. The header schema framework stays :mod:`pcapkit.protocols.schema`. +* **Changed** -- **a breaking change to** the mapping values a caller reads as a + :class:`dict`: each is an :class:`~pcapkit.corekit.infoclass.Info` or a data + model, not a :class:`dict` (:issue:`2726`, :pr:`2734`). + :class:`~pcapkit.foundation.traceflow.data.data.FrameRecord`, the frame a + third-party engine's flow tracing reports, is an ``Info`` where 1.5.x + subclassed :class:`dict`, so ``isinstance(record, dict)`` is false, setting a + key raises :exc:`TypeError`, and ``to_dict()`` exports the engine's mapping + as an :class:`~pcapkit.corekit.multidict.OrderedMultiDict` without the + ``packet``, ``frame_info`` and ``time_epoch`` attributes, which stay + attributes, as in 1.5.x. A parsed TLS key log's + :attr:`TLSKeyLog.entries ` + is a :class:`~pcapkit.protocols.data.misc.pcapng.TLSKeyLogEntries` keyed by + label, which :mod:`pcapkit.protocols.data` exports as + ``PCAPNG_TLSKeyLogEntries``, where it was a :class:`dict`; + a mapping given for it is stored as one, and ``to_dict()`` exports it as an + ``OrderedMultiDict``, where it gave a :class:`dict`. Pickles of either load + across 1.5.x and 1.6.0 in both directions. + * **Changed** -- **a breaking change to** HTTP: parsing octets through :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields and its ``from_data`` give the @@ -883,11 +901,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. an enumeration takes the member or an :class:`int` code, never a name, checked against the field's width and resolved with the ``get`` of an enumeration named at the call, never one read from the caller's frame, and - no keyword group sits beside it (:issue:`2594`, :pr:`2692`). A helper that code outside the schema layer - calls lives in :mod:`pcapkit.corekit.schema`, a module per group, as does a - helper in one protocol's schema module once code outside that protocol uses - it and it is generic enough to reuse, with no re-export at its old path - (:issue:`2674`, :pr:`2704`). + no keyword group sits beside it (:issue:`2594`, :pr:`2692`). A helper that + code outside the schema layer calls lives in :mod:`pcapkit.corekit.schema`, + a module per group, as does a helper in one protocol's schema module once + code outside that protocol uses it and it is generic enough to reuse, with + no re-export at its old path (:issue:`2674`, :pr:`2704`). A protocol reads + and rebuilds a capture cut through the schema's built-in short read, never an + ``unpack`` override of its own or a length callable clamped to the octets + present (:issue:`2680`, :issue:`2694`, :pr:`2731`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -1029,6 +1050,24 @@ HTTP/1 and HTTP/2 classes under the RFC names. schema of data offset 8 cut 25 octets in gave 32 octets from ``bytes()`` and 23 from ``pack()`` (:issue:`2680`, :issue:`2685`, :pr:`2682`). +* **Fixed** -- a capture cut inside or in front of an option area reads as a + short read, as a cut elsewhere in a header does (:issue:`2694`, + :pr:`2731`). So an :class:`~pcapkit.protocols.internet.mh.MH` message cut + inside its options or reserved octets, and a + :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` or + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` header cut short, + parse alone or inside IPv6 and rebuild as captured, logging nothing, where + 1.5.x raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` or + ``ProtocolError`` logged at ``CRITICAL``, as + ``Field options has an option that consumed no data: at offset 0`` + for a HOPOPT header cut to 2 octets. A cut option stands only if its maker + writes the captured octets back, and is refused quietly otherwise; one whose + fields run past its area is refused, as in 1.5.x. HOPOPT's and IPv6-Opts' + ``make()`` take ``len``, the Hdr Ext Len a cut header declared. An MH item + list its length leaves part of an item in, as a Flow Summary option of an + odd length, raises ``ProtocolError``, quiet inside IPv6, where 1.5.x raised + ``FieldValueError: Field fid has invalid length.`` logged at ``CRITICAL``. + * **Fixed** -- :class:`~pcapkit.protocols.transport.tcp.TCP`'s maker, given an option as raw octets, raised :exc:`~pcapkit.utilities.exceptions.EnumValueError` for a kind its registry does not name, such as 255, and :exc:`IndexError` for @@ -1052,7 +1091,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. 1.5.x counted in the parameter's Length but did not write. A parsed ``ENCRYPTED`` parameter takes its cipher from the ``HIP_CIPHER`` before it, where 1.5.x warned ``HIP: [ParamNo 641] missing HIP_CIPHER parameter`` and - read an AES IV as encrypted data. + read an AES IV as encrypted data. MH's CGA Parameters maker builds a raw + extension in a :class:`~pcapkit.protocols.data.internet.mh.CGAParameter` + data model as its ``(type, arguments)`` pair builds, where 1.5.x raised + ``AttributeError: 'bytes' object has no attribute 'type'``, and a + :class:`~pcapkit.protocols.schema.internet.mh.CGAParameter` schema without + ``public_key_test``, leaving the schema given unchanged, where 1.5.x raised + ``KeyError: 'public_key_test'`` (:issue:`2725`, :pr:`2732`). * **Fixed** -- a :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` Decryption Secrets Block that ``make()`` built from a parsed ZigBee NWK or APS key From e915af74deb22d45b82a2d4a7568bce5d8089168 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Tue, 3 Nov 2026 11:51:17 -0500 Subject: [PATCH 23/27] docs(changelog): fold #2735, #2736, #2737, #2741, #2742, #2744 and #2746 - Breaking: a short CALIPSO, SMF_DPD or MPL Opt Data Len raises ProtocolError, where 1.5.x raised FieldValueError, and a cut after the type octet reads short (#2737), a new entry; AppType.__registries__ is an Info, into the #2734 entry (#2735). - Changed: const packages import their enumerations lazily through pcapkit.utilities.lazy.lazy_exports (#2735); the docs extra pins sphinx-autodoc-typehints 3.17.4 (#2742), as 1.5.0 recorded its pin. - Fixed: the dumps keep a field renamed for clashing with an Info builtin name (#2736). - Folded: the GraalPy leg's timeout and shards into the CI-legs entry (#2741); the #2735 and #2737 rulings into the conventions entry. - No entry for the docs-only #2744 and #2746. - 1.6.0's :func:`bytes` role is :class:`bytes`, so the changelog pages build with no nitpicky warning. Old values measured against v1.5.1. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 16 ++++++--- docs/source/changelog/1.6.0.rst | 57 ++++++++++++++++++++++++++++++--- 2 files changed, 65 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 65d83bc0a4..5d90cc0e65 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,6 +46,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. +- **Changed** -- **a breaking change to** parsing a CALIPSO, SMF_DPD or MPL option in `HOPOPT` or `IPv6_Opts`: an Opt Data Len too short for the option raises `ProtocolError`, where 1.5.x raised `FieldValueError`, so code catching `FieldValueError` for it catches `ProtocolError` instead, both being `ValueError`; inside IPv6 the header is kept as `Raw` with nothing logged, where 1.5.x logged it at `CRITICAL` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). So do the length callbacks of `pcapkit.protocols.schema.internet.ipv6_option` that size those options -- `calipso_pad_len`, `mpl_opt_pad_len`, `smf_i_dpd_id_len` and `smf_dpd_data_selector`. A capture cut after such an option's type octet reads as a short read, a CALIPSO or SMF_DPD option kept and rebuilt as captured and an MPL option refused quietly, where 1.5.x raised `FieldValueError: HOPOPT: invalid CALIPSO option length: 0` logged at `CRITICAL`. + - **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. A mobility option whose fields do not take its Length is refused too, and so are such a sub-option, QoS attribute or CGA extension inside one, a target care-of address neither IPv4 nor IPv6, and CGA Parameters whose fixed fields or public key run past the option, where 1.5.x read octets that were not the option's and rebuilt another ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). So an MN-ID option of Length 0 raises `MH: [OptNo 8] invalid option length: 0`, where 1.5.x read its fields from the octets after it and rebuilt it with Length 17, and a Flow Identification option's 8-octet target care-of address raises `MH: [FID SubOpt 5] invalid sub-option length: 10`, where 1.5.x parsed it and raised `FieldValueError` rebuilding it; inside IPv6 the header is `Raw` and the packet rebuilds as captured. - **Changed** -- **a breaking change to** MH option data models. `QualityOfServiceOption`, the Quality-of-Service option's, gains a required `tc_reserved` after `dscp`: the two reserved bits of the traffic class, which 1.5.x dropped, so a traffic class of `0xB9` rebuilt as `0xB8` ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). Building the model without it raises `TypeError`, and `MH`'s `make()` takes it, `0` by default. The MN-ID and Service Selection makers count the octets an identifier packs as, so an NAI of `'usér@x'` takes Length 8, where 1.5.x wrote 7 from its characters, and a service identifier parsed from octets that are not UTF-8 rebuilds as captured, where 1.5.x re-encoded its text, under Length 4 where 3 were captured. A CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses through the long form of its DER length, where 1.5.x misread the length and raised `ProtocolError`. `MobileNetworkPrefixOption`, the Mobile Network Prefix option's, gains a required `prefix_length`, and its `prefix` is the `IPv6Address` captured, where it was an `IPv6Network`, so a prefix with bits set past its length, or a length past 128, parses and rebuilds, where 1.5.x raised a bare `ValueError` such as `2001:db8::1/64 has host bits set` ([#2702](https://github.com/JarryShaw/PyPCAPKit/issues/2702), [#2709](https://github.com/JarryShaw/PyPCAPKit/pull/2709)). Its maker takes `prefix_length`, from 0 to 255, where 1.5.x ignored it and wrote the length `prefix` gave, 128 for an address; and it takes `prefix` as an address, a network or an `int`, keeping any bits past the length. In `TimestampOption`, `timestamp` is `None` past the year 9999, `pmip_timestamp` holding the value, where 1.5.x raised `ValueError` for a time past it, or late enough in its last second to round up to the year 10000. It is read without summing the second and its fraction as a `float`, which was off by up to 15 microseconds past `2**36` seconds. @@ -68,7 +70,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** the schema helpers that code outside the schema layer calls, which move to `pcapkit.corekit.schema`, a package with a module per group, with no re-export at their old paths ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). `parse_ip_address()` moves from `pcapkit.corekit.fields.ipaddress` to `pcapkit.corekit.schema.ipaddress`; `packet_byteorder()` and `byteorder_callback()` from `pcapkit.corekit.fields.byteorder`, which is removed with no alias, to `pcapkit.corekit.schema.byteorder`; `jumbo_payload_length()` from `pcapkit.protocols.schema.internet.ipv6` to `pcapkit.corekit.schema.jumbogram`; and `keep_short_read()` and `replay_short_read()` from `pcapkit.protocols.schema.schema` to `pcapkit.corekit.schema.shortread`. The header schema framework stays `pcapkit.protocols.schema`. -- **Changed** -- **a breaking change to** the mapping values a caller reads as a `dict`: each is an `Info` or a data model, not a `dict` ([#2726](https://github.com/JarryShaw/PyPCAPKit/issues/2726), [#2734](https://github.com/JarryShaw/PyPCAPKit/pull/2734)). `FrameRecord`, the frame a third-party engine's flow tracing reports, is an `Info` where 1.5.x subclassed `dict`, so `isinstance(record, dict)` is false, setting a key raises `TypeError`, and `to_dict()` exports the engine's mapping as an `OrderedMultiDict` without the `packet`, `frame_info` and `time_epoch` attributes, which stay attributes, as in 1.5.x. A parsed TLS key log's `TLSKeyLog.entries` is a `TLSKeyLogEntries` keyed by label, which `pcapkit.protocols.data` exports as `PCAPNG_TLSKeyLogEntries`, where it was a `dict`; a mapping given for it is stored as one, and `to_dict()` exports it as an `OrderedMultiDict`, where it gave a `dict`. Pickles of either load across 1.5.x and 1.6.0 in both directions. +- **Changed** -- **a breaking change to** the mapping values a caller reads as a `dict`: each is an `Info` or a data model, not a `dict` ([#2726](https://github.com/JarryShaw/PyPCAPKit/issues/2726), [#2734](https://github.com/JarryShaw/PyPCAPKit/pull/2734)). `FrameRecord`, the frame a third-party engine's flow tracing reports, is an `Info` where 1.5.x subclassed `dict`, so `isinstance(record, dict)` is false, setting a key raises `TypeError`, and `to_dict()` exports the engine's mapping as an `OrderedMultiDict` without the `packet`, `frame_info` and `time_epoch` attributes, which stay attributes, as in 1.5.x. A parsed TLS key log's `TLSKeyLog.entries` is a `TLSKeyLogEntries` keyed by label, which `pcapkit.protocols.data` exports as `PCAPNG_TLSKeyLogEntries`, where it was a `dict`; a mapping given for it is stored as one, and `to_dict()` exports it as an `OrderedMultiDict`, where it gave a `dict`. Pickles of either load across 1.5.x and 1.6.0 in both directions. `AppType.__registries__` is an `Info` keyed by `TransportProtocol`, where 1.5.x held a `dict` subclass, so setting a key raises `TypeError` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). Building it from fields takes `version`, `1` or `2`, as the other versioned bases above do, and raises `ProtocolError` without it, where 1.5.x built an HTTP/1.1 message; its `from_data` takes the member from the data model's class and refuses a `dict`, where 1.5.x took one ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`, which `HTTPv1` refuses, where 1.5.x wrote the HTTP/2 preface ahead of the HTTP/1 message ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). @@ -80,17 +82,21 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- `pcapkit.foundation.registry.protocols` is a package with a submodule per layer -- `link`, `internet`, `transport`, `application` and `misc`, with the shared registrars in `protocol` -- and exports every public function they define, so each `register_*` imports from it and from `pcapkit.foundation.registry` as before ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A registrar's `__module__` names its submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. +- **Changed** -- a `pcapkit.const` package imports each of its enumeration modules on first use rather than with itself, and `pcapkit.const` its packages, through `pcapkit.utilities.lazy.lazy_exports()`, which builds a package's PEP 562 `__getattr__` and `__dir__` from its exports ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). Each package's `__all__`, `dir()` and members are as in 1.5.x. The SCTP and DCCP application-type registries load on first use, as TCP's and UDP's did in 1.5.x, where importing `pcapkit.const` loaded both. + - **Changed** -- `SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as `SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''` and `SCTP: invalid [Cause 1] reserved: must be 2 octets, not b''`, where they read `SCTP: [Chunk 132] invalid flags: must be one octet, not b''` and `SCTP: [Cause 1] invalid reserved: must be two octets, not b''` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A one-character `str` given as flags, which passed the check and failed to pack, is refused the same way. - **Changed** -- a protocol's `__index__` returns its value in the LinkType, EtherType, TransType or AppType registry wherever one names it, and a class an AppType member names returns that member's port as an `int` ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). So `L2TPv2`, `FTP`, `FTP_DATA`, the HTTP classes and `NGAP`, which raised in 1.5.x, return 1701, 21, 20, 80 and 38412, and TLS, SSLv3 and DTLS the ports TCP and UDP dispatch them at, 443 and 5684; Radiotap, FRAD and GRE return their LinkType or TransType member. SSLv2 and the SSL base, which nothing dispatches to, raise as before, and no value 1.5.x returned changes. - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). The GraalPy leg there splits the long shards whose xdist workers outgrew their heap and died, each sized from its tests' JUnit seconds to run in under 5 minutes ([#2713](https://github.com/JarryShaw/PyPCAPKit/issues/2713), [#2720](https://github.com/JarryShaw/PyPCAPKit/pull/2720)). +- **Changed** -- the `docs` extra pins `sphinx-autodoc-typehints` 3.17.4, where 1.5.x pinned 3.17.2 ([#2742](https://github.com/JarryShaw/PyPCAPKit/pull/2742)). + +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). The GraalPy leg there splits the long shards whose xdist workers outgrew their heap and died, each sized from its tests' JUnit seconds to run in under 5 minutes ([#2713](https://github.com/JarryShaw/PyPCAPKit/issues/2713), [#2720](https://github.com/JarryShaw/PyPCAPKit/pull/2720)). Its tests' timeout is 180 minutes, where it was 110, and the internet and application protocol shards are split too, as a slower runner ran out of time with no worker lost ([#2739](https://github.com/JarryShaw/PyPCAPKit/issues/2739), [#2741](https://github.com/JarryShaw/PyPCAPKit/pull/2741)). - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). @@ -122,7 +128,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). The other makers taking a raw item refuse empty octets the same way, as `IPv4: invalid option: b'' holds no type octet` ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). IPv4's, HOPOPT's and IPv6-Opts' options raised `IndexError`. SCTP's parameters and error causes, and MH's options, sub-options, QoS attributes and CGA extensions, raised a bare `AttributeError` alone -- or, for MH's access network identifier and LMA-controlled MAG sub-options, a `ProtocolError` for an option of length 0 -- and were dropped beside a whole item. HIP's parameters were dropped. PCAP-NG's options and Name Resolution records refuse octets shorter than their 2-octet code, where 1.5.x wrote empty or zero octets as an end-of-options or end-of-records marker, and misread one other octet as a code. `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. MH's CGA Parameters maker builds a raw extension in a `CGAParameter` data model as its `(type, arguments)` pair builds, where 1.5.x raised `AttributeError: 'bytes' object has no attribute 'type'`, and a `CGAParameter` schema without `public_key_test`, leaving the schema given unchanged, where 1.5.x raised `KeyError: 'public_key_test'` ([#2725](https://github.com/JarryShaw/PyPCAPKit/issues/2725), [#2732](https://github.com/JarryShaw/PyPCAPKit/pull/2732)). -- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes()` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). +- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). + +- **Fixed** -- the `json`, `plist` and `tree` dumps of an `Info`, and of its `to_dict()` export, left out a field named like one of its builtin names, such as `items`, `keys` or a reassembled datagram's `packet`. Each is dumped under its own name, and a reassembled datagram's `packet` as its parsed layer's `info` ([#2733](https://github.com/JarryShaw/PyPCAPKit/issues/2733), [#2736](https://github.com/JarryShaw/PyPCAPKit/pull/2736)). - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 8979c45e61..8f2b868942 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -466,6 +466,23 @@ HTTP/1 and HTTP/2 classes under the RFC names. is :data:`None` unless a caller supplies one; the ``Raw`` layer's ``info.error`` holds the failure instead. +* **Changed** -- **a breaking change to** parsing a CALIPSO, SMF_DPD or MPL + option in :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` or + :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts`: an Opt Data Len too + short for the option raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, + where 1.5.x raised :exc:`~pcapkit.utilities.exceptions.FieldValueError`, so + code catching ``FieldValueError`` for it catches ``ProtocolError`` instead, + both being :exc:`ValueError`; inside IPv6 the header is kept as ``Raw`` with + nothing logged, where 1.5.x logged it at ``CRITICAL`` (:issue:`2727`, + :pr:`2737`). So do the length + callbacks of :mod:`pcapkit.protocols.schema.internet.ipv6_option` that size + those options -- ``calipso_pad_len``, ``mpl_opt_pad_len``, + ``smf_i_dpd_id_len`` and ``smf_dpd_data_selector``. A capture cut after such + an option's type octet reads as a short read, a CALIPSO or SMF_DPD option + kept and rebuilt as captured and an MPL option refused quietly, where 1.5.x + raised ``FieldValueError: HOPOPT: invalid CALIPSO option length: 0`` logged + at ``CRITICAL``. + * **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x read the @@ -736,7 +753,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``PCAPNG_TLSKeyLogEntries``, where it was a :class:`dict`; a mapping given for it is stored as one, and ``to_dict()`` exports it as an ``OrderedMultiDict``, where it gave a :class:`dict`. Pickles of either load - across 1.5.x and 1.6.0 in both directions. + across 1.5.x and 1.6.0 in both directions. ``AppType.__registries__`` is an + ``Info`` keyed by + :class:`~pcapkit.const.reg.apptype.apptype.TransportProtocol`, where 1.5.x + held a :class:`dict` subclass, so setting a key raises :exc:`TypeError` + (:issue:`2729`, :pr:`2735`). * **Changed** -- **a breaking change to** HTTP: parsing octets through :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields @@ -811,6 +832,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. +* **Changed** -- a :mod:`pcapkit.const` package imports each of its + enumeration modules on first use rather than with itself, and + :mod:`pcapkit.const` its packages, through + :func:`pcapkit.utilities.lazy.lazy_exports`, which builds a package's + PEP 562 ``__getattr__`` and ``__dir__`` from its exports + (:issue:`2729`, :pr:`2735`). Each package's ``__all__``, ``dir()`` and + members are as in 1.5.x. The SCTP and DCCP application-type registries load + on first use, as TCP's and UDP's did in 1.5.x, where importing + :mod:`pcapkit.const` loaded both. + * **Changed** -- :class:`~pcapkit.protocols.transport.sctp.SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as @@ -908,7 +939,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. no re-export at its old path (:issue:`2674`, :pr:`2704`). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an ``unpack`` override of its own or a length callable clamped to the octets - present (:issue:`2680`, :issue:`2694`, :pr:`2731`). + present (:issue:`2680`, :issue:`2694`, :pr:`2731`); a field callback leaves + a value the cut left as zero to that short read, and refuses one read with + ``ProtocolError`` (:issue:`2727`, :pr:`2737`). A :mod:`pcapkit.const` + package imports its enumerations on first use, through + :func:`~pcapkit.utilities.lazy.lazy_exports`, and a crawler never writes a + package's ``__init__.py`` (:issue:`2729`, :pr:`2735`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -917,6 +953,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one (:issue:`2559`, :pr:`2584`). +* **Changed** -- the ``docs`` extra pins ``sphinx-autodoc-typehints`` 3.17.4, + where 1.5.x pinned 3.17.2 (:pr:`2742`). + * **Changed** -- the CPython unit-test legs run ``pytest -n logical``, a worker per logical CPU, where ``-n auto`` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's @@ -933,7 +972,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. outgrown Windows' limit, so the leg failed before pytest started (:issue:`2678`, :pr:`2681`). The GraalPy leg there splits the long shards whose xdist workers outgrew their heap and died, each sized from its tests' - JUnit seconds to run in under 5 minutes (:issue:`2713`, :pr:`2720`). + JUnit seconds to run in under 5 minutes (:issue:`2713`, :pr:`2720`). Its + tests' timeout is 180 minutes, where it was 110, and the internet and + application protocol shards are split too, as a slower runner ran out of + time with no worker lost (:issue:`2739`, :pr:`2741`). * **Changed** -- once a release's milestone is closed while ``__version__`` still names that release, the milestone workflow gives an item the lowest @@ -1106,7 +1148,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and ``pcapkit.protocols.schema.misc.pcapng.zigbee_key_held`` returns a schema - whose :func:`bytes` and :func:`len` are those of the octets held, where they + whose :class:`bytes` and :func:`len` are those of the octets held, where they were the fixed width's (:issue:`2688`, :pr:`2697`). Rebuilt by ``from_data`` from the block's :class:`dict`, a TLS or WireGuard key log was written from its entries under a ``# generated by PyPCAPKit`` line, dropping its comments and @@ -1116,6 +1158,13 @@ HTTP/1 and HTTP/2 classes under the RFC names. rebuilds as captured from its :class:`dict`, as from its data model (:issue:`2698`, :pr:`2707`). +* **Fixed** -- the ``json``, ``plist`` and ``tree`` dumps of an + :class:`~pcapkit.corekit.infoclass.Info`, and of its ``to_dict()`` export, + left out a field named like one of its builtin names, such as ``items``, + ``keys`` or a reassembled datagram's ``packet``. Each is dumped under its own + name, and a reassembled datagram's ``packet`` as its parsed layer's ``info`` + (:issue:`2733`, :pr:`2736`). + * **Fixed** -- importing :mod:`pcapkit.vendor.ipx`, or running its packet or socket crawler, without ``requests`` installed lost an entry of :data:`sys.path` on each failed attempt, so a :func:`pkgutil.walk_packages` From fb64180eba58b135256182a914902ea13172ba76 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Tue, 3 Nov 2026 15:13:22 -0500 Subject: [PATCH 24/27] docs(changelog): fold #2738 and the utilities.importlib rename - Fixed: a layer rebuilt by from_data reports its payload's messages once, where 1.5.x logged them twice, and IPv4 and the frame-end layers record a cut as their own short read (#2738), a new entry; old values measured against v1.5.1. - Folded into unreleased entries (#2738): a standalone IGMPv3 Report sized by its record count, into the IGMP entry; the recorded cuts' TRUNCATED status, TSO IPv4 and ICMPv4 included, into the checksum entry; ListField's count, into its zero-fill entry; the quiet rebuilt payload, into the conventions entry. - #2748: pcapkit.utilities.lazy becomes pcapkit.utilities.importlib in the lazy-import and conventions entries; the old name appears nowhere. - No entry for the direct-pushed 21ec0a6f1, which has no pull request or issue to cite. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 12 +++++++----- docs/source/changelog/1.6.0.rst | 34 +++++++++++++++++++++++++++------ 2 files changed, 35 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5d90cc0e65..c5ffe14517 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). The octets it lacks read as zeros, the group records included, and a Report's header schema cut inside the Additional Data past its records packs back as captured ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2687](https://github.com/JarryShaw/PyPCAPKit/issues/2687), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). The octets it lacks read as zeros, the group records included, and a Report's header schema cut inside the Additional Data past its records packs back as captured ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2687](https://github.com/JarryShaw/PyPCAPKit/issues/2687), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). A Report parsed on its own sizes its group records by its record count, so one cut before the records it counts reads `TRUNCATED`, as it does behind IPv4, holding the records captured and none made up, and a count no IP payload could carry raises `ProtocolError` ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. - **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. HTTP's base follows the same rules, a breaking change given below ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. @@ -34,7 +34,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `IP_Pseudo`, the IPv4 ([RFC 793](https://datatracker.ietf.org/doc/html/rfc793)) or IPv6 ([RFC 8200 Section 8.1](https://datatracker.ietf.org/doc/html/rfc8200#section-8.1)) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both ([#2471](https://github.com/JarryShaw/PyPCAPKit/issues/2471), [#2479](https://github.com/JarryShaw/PyPCAPKit/pull/2479)). From a layer it takes the final destination of a source route or of a Routing header with segments left, a Home Address option's address as the IPv6 source, and the upper-layer length, from UDP's Length or a jumbogram's Jumbo Payload Length where those apply; a fragment of a larger packet, or a header it cannot walk at any depth of the chain, raises `ProtocolError` ([#2513](https://github.com/JarryShaw/PyPCAPKit/issues/2513), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)). It is not a layer on the wire. -- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). ICMP's, IGMP's and GRE's checksums take those lengths as well behind an IPv4 Total Length of 0, as TCP segmentation offload leaves it, and behind an AH there, so one the capture cut short is `TRUNCATED` with no warning; a GRE or IGMP layer read directly from octets that end inside its header or message is `TRUNCATED` too ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). A layer `from_data` rebuilds from a cut capture, from its data model or its `dict`, verifies as its parse did, so a cut GRE, ICMP, IGMP or OSPF layer is `TRUNCATED` with no warning ([#2703](https://github.com/JarryShaw/PyPCAPKit/issues/2703), [#2712](https://github.com/JarryShaw/PyPCAPKit/pull/2712)). +- **Added** -- verification and computation of the one's-complement Internet checksum ([RFC 1071](https://datatracker.ietf.org/doc/html/rfc1071)) of the IPv4 header, TCP, UDP, ICMP, ICMPv6 and OSPFv2, in `pcapkit.corekit.checksum.internet` ([#2470](https://github.com/JarryShaw/PyPCAPKit/issues/2470), [#2472](https://github.com/JarryShaw/PyPCAPKit/issues/2472), [#2473](https://github.com/JarryShaw/PyPCAPKit/issues/2473), [#2474](https://github.com/JarryShaw/PyPCAPKit/issues/2474), [#2475](https://github.com/JarryShaw/PyPCAPKit/issues/2475), [#2543](https://github.com/JarryShaw/PyPCAPKit/pull/2543)), and of IGMP and GRE ([#2618](https://github.com/JarryShaw/PyPCAPKit/issues/2618), [#2654](https://github.com/JarryShaw/PyPCAPKit/pull/2654)), where `pcapkit.corekit.checksum` holds a module per checksum family and imports nothing from `pcapkit.protocols` ([#2580](https://github.com/JarryShaw/PyPCAPKit/issues/2580), [#2586](https://github.com/JarryShaw/PyPCAPKit/pull/2586)). Parsing verifies only under `PCAPKIT_CHECKSUM_VERIFY`, off by default, recording a `ChecksumStatus` as `checksum_status` and warning on a mismatch with `ChecksumWarning`, a `ProtocolWarning` a filter can silence alone. Construction computes a checksum the caller leaves out, unless `PCAPKIT_CHECKSUM_COMPUTE` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the `IP_Pseudo` it builds, and `make()` of TCP, UDP and ICMPv6 also takes a `pseudo`. What checksum offload leaves on the sending host is `NOT_COMPUTED` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which covers octets the fragment lacks, is `NO_PSEUDO_HEADER`. `verify_message()` verifies a checksum that covers its own message alone, as ICMP's, IGMP's and GRE's do, giving `NO_PSEUDO_HEADER` for a fragment's payload. GRE verifies only with its `C` bit set, and is `NOT_COMPUTED` with it clear; its `make()` takes `checksum_present` to set the bit and compute the checksum. GRE states no length, so outside an IP layer, which states one, it takes the captured and original lengths of its frame, and a GRE layer the capture cut short is `TRUNCATED` either way ([#2653](https://github.com/JarryShaw/PyPCAPKit/issues/2653), [#2668](https://github.com/JarryShaw/PyPCAPKit/pull/2668)). ICMP's, IGMP's and GRE's checksums take those lengths as well behind an IPv4 Total Length of 0, as TCP segmentation offload leaves it, and behind an AH there, so one the capture cut short is `TRUNCATED` with no warning; a GRE or IGMP layer read directly from octets that end inside its header or message is `TRUNCATED` too ([#2689](https://github.com/JarryShaw/PyPCAPKit/issues/2689), [#2699](https://github.com/JarryShaw/PyPCAPKit/pull/2699)). A layer `from_data` rebuilds from a cut capture, from its data model or its `dict`, verifies as its parse did, so a cut GRE, ICMP, IGMP or OSPF layer is `TRUNCATED` with no warning ([#2703](https://github.com/JarryShaw/PyPCAPKit/issues/2703), [#2712](https://github.com/JarryShaw/PyPCAPKit/pull/2712)). Each of these cuts, a TSO IPv4 payload's included, is recorded on the message's own header, so the message rebuilt on its own reads `TRUNCATED` too, an ICMPv4 one included ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). - **Added** -- `register_ipx()`, `register_loopback()`, `register_sctp_chunk()`, `register_sctp_parameter()` and `register_sctp_cause()`, wrapping IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the `IPX.register` and `Loopback.register` classmethods the IPX and Loopback wrappers call ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). @@ -82,7 +82,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- `pcapkit.foundation.registry.protocols` is a package with a submodule per layer -- `link`, `internet`, `transport`, `application` and `misc`, with the shared registrars in `protocol` -- and exports every public function they define, so each `register_*` imports from it and from `pcapkit.foundation.registry` as before ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A registrar's `__module__` names its submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. -- **Changed** -- a `pcapkit.const` package imports each of its enumeration modules on first use rather than with itself, and `pcapkit.const` its packages, through `pcapkit.utilities.lazy.lazy_exports()`, which builds a package's PEP 562 `__getattr__` and `__dir__` from its exports ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). Each package's `__all__`, `dir()` and members are as in 1.5.x. The SCTP and DCCP application-type registries load on first use, as TCP's and UDP's did in 1.5.x, where importing `pcapkit.const` loaded both. +- **Changed** -- a `pcapkit.const` package imports each of its enumeration modules on first use rather than with itself, and `pcapkit.const` its packages, through `pcapkit.utilities.importlib.lazy_exports()`, which builds a package's PEP 562 `__getattr__` and `__dir__` from its exports ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). Each package's `__all__`, `dir()` and members are as in 1.5.x. The SCTP and DCCP application-type registries load on first use, as TCP's and UDP's did in 1.5.x, where importing `pcapkit.const` loaded both. - **Changed** -- `SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as `SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''` and `SCTP: invalid [Cause 1] reserved: must be 2 octets, not b''`, where they read `SCTP: [Chunk 132] invalid flags: must be one octet, not b''` and `SCTP: [Cause 1] invalid reserved: must be two octets, not b''` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A one-character `str` given as flags, which passed the check and failed to pack, is refused the same way. @@ -90,7 +90,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). +- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). A next-layer refusal in a payload a layer rebuilds is quiet, as the layer's own parse reports it once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). @@ -118,7 +118,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). -- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside zero-filled to its length, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Each item filled is a schema object rather than a zero octet, so `ZERO_PAD_BUDGET_RATIO` bounds the fill from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. If the items then fail, the fill stays charged, as a field's does, and the octets it counted as read are given back ([#2705](https://github.com/JarryShaw/PyPCAPKit/issues/2705), [#2706](https://github.com/JarryShaw/PyPCAPKit/pull/2706)). A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. +- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside zero-filled to its length, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Each item filled is a schema object rather than a zero octet, so `ZERO_PAD_BUDGET_RATIO` bounds the fill from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. If the items then fail, the fill stays charged, as a field's does, and the octets it counted as read are given back ([#2705](https://github.com/JarryShaw/PyPCAPKit/issues/2705), [#2706](https://github.com/JarryShaw/PyPCAPKit/pull/2706)). A `ListField` given a `count` reports a list whose octets hold fewer items than it counts as a short read, the absent items never made up ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. - **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses, with the octets its settings lack read as zeros, as a cut `DATA` frame's data is, and rebuilds as captured ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). @@ -130,6 +130,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). +- **Fixed** -- rebuilding a layer through `from_data` reported its payload's messages twice, once from the payload's own rebuild and again from the layer's parse of it: in 1.5.x, `Ethernet.from_data` of a frame cut inside its TCP header logged the refusal twice, where the parse logged it once. Each is reported once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). IPv4 records a cut inside the payload length it declares as its `__short_read__`, where 1.5.x recorded none, and so do Ethernet, GRE, ICMP, IGMP and a TSO IPv4 layer for a cut at the end of their frame. + - **Fixed** -- the `json`, `plist` and `tree` dumps of an `Info`, and of its `to_dict()` export, left out a field named like one of its builtin names, such as `items`, `keys` or a reassembled datagram's `packet`. Each is dumped under its own name, and a reassembled datagram's `packet` as its parsed layer's `info` ([#2733](https://github.com/JarryShaw/PyPCAPKit/issues/2733), [#2736](https://github.com/JarryShaw/PyPCAPKit/pull/2736)). - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 8f2b868942..8d88dd021f 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -187,7 +187,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``0xFFFF`` (:issue:`2650`, :issue:`2660`, :pr:`2662`, :pr:`2672`). The octets it lacks read as zeros, the group records included, and a Report's header schema cut inside the Additional Data past its records packs back as - captured (:issue:`2686`, :issue:`2687`, :pr:`2701`). Any other message, a 9- to + captured (:issue:`2686`, :issue:`2687`, :pr:`2701`). A Report parsed on its + own sizes its group records by its record count, so one cut before the + records it counts reads ``TRUNCATED``, as it does behind IPv4, holding the + records captured and none made up, and a count no IP payload could carry + raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2711`, + :pr:`2738`). Any other message, a 9- to 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and @@ -403,7 +408,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``TRUNCATED`` too (:issue:`2689`, :pr:`2699`). A layer ``from_data`` rebuilds from a cut capture, from its data model or its :class:`dict`, verifies as its parse did, so a cut GRE, ICMP, IGMP or OSPF layer is ``TRUNCATED`` with no - warning (:issue:`2703`, :pr:`2712`). + warning (:issue:`2703`, :pr:`2712`). Each of these cuts, a TSO IPv4 + payload's included, is recorded on the message's own header, so the message + rebuilt on its own reads ``TRUNCATED`` too, an ICMPv4 one included + (:issue:`2711`, :pr:`2738`). * **Added** -- :func:`~pcapkit.foundation.registry.protocols.internet.register_ipx`, :func:`~pcapkit.foundation.registry.protocols.link.register_loopback`, @@ -835,7 +843,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- a :mod:`pcapkit.const` package imports each of its enumeration modules on first use rather than with itself, and :mod:`pcapkit.const` its packages, through - :func:`pcapkit.utilities.lazy.lazy_exports`, which builds a package's + :func:`pcapkit.utilities.importlib.lazy_exports`, which builds a package's PEP 562 ``__getattr__`` and ``__dir__`` from its exports (:issue:`2729`, :pr:`2735`). Each package's ``__all__``, ``dir()`` and members are as in 1.5.x. The SCTP and DCCP application-type registries load @@ -943,8 +951,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. a value the cut left as zero to that short read, and refuses one read with ``ProtocolError`` (:issue:`2727`, :pr:`2737`). A :mod:`pcapkit.const` package imports its enumerations on first use, through - :func:`~pcapkit.utilities.lazy.lazy_exports`, and a crawler never writes a - package's ``__init__.py`` (:issue:`2729`, :pr:`2735`). + :func:`~pcapkit.utilities.importlib.lazy_exports`, and a crawler never + writes a package's ``__init__.py`` (:issue:`2729`, :pr:`2735`). A next-layer + refusal in a payload a layer rebuilds is quiet, as the layer's own parse + reports it once (:issue:`2711`, :pr:`2738`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -1062,7 +1072,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. allowance, and a cut list declaring more than :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_LENGTH` is refused. If the items then fail, the fill stays charged, as a field's does, and the octets - it counted as read are given back (:issue:`2705`, :pr:`2706`). A TCP + it counted as read are given back (:issue:`2705`, :pr:`2706`). A + ``ListField`` given a ``count`` reports a list whose octets hold fewer items + than it counts as a short read, the absent items never made up + (:issue:`2711`, :pr:`2738`). A TCP ``SACK`` option running past the data raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, ``Field options has an option that runs past the end of the data: ...``, @@ -1158,6 +1171,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. rebuilds as captured from its :class:`dict`, as from its data model (:issue:`2698`, :pr:`2707`). +* **Fixed** -- rebuilding a layer through ``from_data`` reported its payload's + messages twice, once from the payload's own rebuild and again from the + layer's parse of it: in 1.5.x, ``Ethernet.from_data`` of a frame cut inside + its TCP header logged the refusal twice, where the parse logged it once. + Each is reported once (:issue:`2711`, :pr:`2738`). IPv4 records a cut inside + the payload length it declares as its ``__short_read__``, where 1.5.x + recorded none, and so do Ethernet, GRE, ICMP, IGMP and a TSO IPv4 layer for + a cut at the end of their frame. + * **Fixed** -- the ``json``, ``plist`` and ``tree`` dumps of an :class:`~pcapkit.corekit.infoclass.Info`, and of its ``to_dict()`` export, left out a field named like one of its builtin names, such as ``items``, From 3bd890f2843ba4a596da9740ccac82c4938db570 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Tue, 3 Nov 2026 22:26:45 -0500 Subject: [PATCH 25/27] docs(changelog): fold #2752, #2753, #2754 and the ordering split - Breaking: each protocol is a package of protocol, schema and data modules, the pcapkit.protocols.schema and .data trees and their aggregate exports removed with no alias, and a 1.5.x data model pickle fails to load (#2753), a new entry with the 1.5.1 imports and pickle measured. - 1.6.0's pcapkit.protocols role targets point at the package modules; the aggregator export claims of unreleased entries, the NoPayload and FrameRecord pickle claims and the TLSKeyLogEntries alias follow the layout. - CI: the LATEST tier's own job and the Coverage merge (#2754) and the ordering leg's halved rest of tests/protocols (21ec0a6f1, #2751), into the CI-legs entry; the package ruling into the conventions entry. - No entry for the test-only #2752. - The merge keeps #2483's class names in 1.5.0's conflicting roles under #2753's package paths. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 20 +- docs/source/changelog/1.6.0.rst | 319 ++++++++++++++++++-------------- 2 files changed, 189 insertions(+), 150 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c5ffe14517..05f4a92a17 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,7 +10,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `GRE`, a dissector for the Generic Routing Encapsulation base header ([RFC 2784](https://datatracker.ietf.org/doc/html/rfc2784)) plus the optional Key and Sequence Number extensions ([RFC 2890](https://datatracker.ietf.org/doc/html/rfc2890)), which `LinuxSLL` dispatches to for both `ARPHRD_IPGRE` and `ARPHRD_IP6GRE` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version ([#2501](https://github.com/JarryShaw/PyPCAPKit/issues/2501), [#2508](https://github.com/JarryShaw/PyPCAPKit/pull/2508)), as does an IP layer for IP protocol 47, through the registry (`Internet.__proto__`) IPv4 and IPv6 share ([#2649](https://github.com/JarryShaw/PyPCAPKit/issues/2649), [#2667](https://github.com/JarryShaw/PyPCAPKit/pull/2667)). So with reassembly and tracing on, analysis follows a GRE tunnel: TCP inside GRE over IP reassembles under its inner addresses, and a fragmented GRE datagram reassembles to a parsed GRE layer. The Protocol Type field dispatches onward as an EtherType, through the family registry (`Link.__proto__`) Ethernet, VLAN and LinuxSLL share. A non-zero Version, [RFC 2637](https://datatracker.ietf.org/doc/html/rfc2637)'s Enhanced GRE, is kept as `Raw` rather than misread. -- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole as `fields`, which a rebuild writes, and decoded as `decoded`, an entry in wire order for each bit that selects a radiotap field or switches namespace, each field aligned from the start of the header as radiotap.org lays it out ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). That covers radiotap.org's defined fields, the RTS and data retries of bits 16 and 17, the radiotap and vendor namespace switches, a vendor namespace's data kept raw, and the items of TLV mode, a vendor item split into its header and raw data. An item a producer cut short of its layout, as radiotap.org allows, decodes with the octets it lacks as zeros and rebuilds as captured ([#2655](https://github.com/JarryShaw/PyPCAPKit/issues/2655), [#2690](https://github.com/JarryShaw/PyPCAPKit/pull/2690)). A bit no field is defined for, or a field running past the region, ends the walk as an `Unknown` entry holding the rest, and `make()` takes the fields decoded as well as raw. Each field type has a concrete `RadiotapField` schema, and is read and made through `Radiotap.__field__`, which `Radiotap.register_field` and `register_radiotap_field()` extend; `pcapkit.protocols.schema` and `pcapkit.protocols.data` export the schemas and data models with a `Radiotap_` prefix. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which keeps the bits it does not name. One mixing value subfields with flags -- MCS `flags`, HE `data1` and `data2`, HE-MU `flags1` and `flags2` and U-SIG `common` -- decodes into a data model of its subfields, each an `int`, and, except in MCS's, its other bits as `known`, a member of a known-bits registry such as `HEData1Known`; `make()` takes it as a mapping, refusing a subfield its bits cannot hold or a `known` setting a subfield's bit with `ProtocolError` ([#2647](https://github.com/JarryShaw/PyPCAPKit/issues/2647), [#2696](https://github.com/JarryShaw/PyPCAPKit/pull/2696)). `BitmapField` reads such a word, an integer in its byte order split into subfields counted from its least-significant bit. The field types and those registries are `pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag takes the name of its macro in Linux's `ieee80211_radiotap.h`, or in Wireshark where Linux defines none, and radiotap.org's wording where neither does. The 802.11 MAC frame behind the header is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). +- **Added** -- `Radiotap`, a dissector for the radiotap header () capture tools prepend to an 802.11 frame, which `LinuxSLL` dispatches to for `ARPHRD_IEEE80211_RADIOTAP` ([#2502](https://github.com/JarryShaw/PyPCAPKit/issues/2502), [#2509](https://github.com/JarryShaw/PyPCAPKit/pull/2509)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11_RADIOTAP` (127) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It decodes the fixed 8-octet start (`it_version`, `it_pad`, `it_len`) and records every present-flags word, following bit 31 from one word to the next. The optional fields those words select are kept whole as `fields`, which a rebuild writes, and decoded as `decoded`, an entry in wire order for each bit that selects a radiotap field or switches namespace, each field aligned from the start of the header as radiotap.org lays it out ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). That covers radiotap.org's defined fields, the RTS and data retries of bits 16 and 17, the radiotap and vendor namespace switches, a vendor namespace's data kept raw, and the items of TLV mode, a vendor item split into its header and raw data. An item a producer cut short of its layout, as radiotap.org allows, decodes with the octets it lacks as zeros and rebuilds as captured ([#2655](https://github.com/JarryShaw/PyPCAPKit/issues/2655), [#2690](https://github.com/JarryShaw/PyPCAPKit/pull/2690)). A bit no field is defined for, or a field running past the region, ends the walk as an `Unknown` entry holding the rest, and `make()` takes the fields decoded as well as raw. Each field type has a concrete `RadiotapField` schema, and is read and made through `Radiotap.__field__`, which `Radiotap.register_field` and `register_radiotap_field()` extend. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which keeps the bits it does not name. One mixing value subfields with flags -- MCS `flags`, HE `data1` and `data2`, HE-MU `flags1` and `flags2` and U-SIG `common` -- decodes into a data model of its subfields, each an `int`, and, except in MCS's, its other bits as `known`, a member of a known-bits registry such as `HEData1Known`; `make()` takes it as a mapping, refusing a subfield its bits cannot hold or a `known` setting a subfield's bit with `ProtocolError` ([#2647](https://github.com/JarryShaw/PyPCAPKit/issues/2647), [#2696](https://github.com/JarryShaw/PyPCAPKit/pull/2696)). `BitmapField` reads such a word, an integer in its byte order split into subfields counted from its least-significant bit. The field types and those registries are `pcapkit.const.radiotap`, crawled from radiotap.org's pages; a flag takes the name of its macro in Linux's `ieee80211_radiotap.h`, or in Wireshark where Linux defines none, and radiotap.org's wording where neither does. The 802.11 MAC frame behind the header is dissected as `IEEE802_11`, below, looked up in `Radiotap`'s own registry under `LinkType.IEEE802_11`, and `register_radiotap()` replaces that dissector ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). - **Added** -- `IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, which `Radiotap` dispatches to ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)), as do PCAP and PCAP-NG frames of `LINKTYPE_IEEE802_11` (105) ([#2608](https://github.com/JarryShaw/PyPCAPKit/issues/2608), [#2632](https://github.com/JarryShaw/PyPCAPKit/pull/2632)). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal header. The frame body is kept as `Raw`, with the FCS and radiotap's pad octets where captured, as `Radiotap` decodes the Flags that mark them but does not pass them on, and any other protocol version is refused. The frame types and subtypes are `pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. @@ -22,7 +22,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `DCCP` ([RFC 4340](https://datatracker.ietf.org/doc/html/rfc4340)) for IP protocol 33, which was kept as `Raw` ([#2489](https://github.com/JarryShaw/PyPCAPKit/issues/2489), [#2593](https://github.com/JarryShaw/PyPCAPKit/pull/2593)). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen ([RFC 5596](https://datatracker.ietf.org/doc/html/rfc5596)) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its checksum is verified and computed as TCP's is, below, over the enclosing IP layer's pseudo-header and the part of the packet its Checksum Coverage gives ([RFC 4340 Section 9.2](https://datatracker.ietf.org/doc/html/rfc4340#section-9.2)): a coverage running past the packet is `INVALID` and warns, and a `make()` that computes the checksum writes a coverage reaching the end of the packet as 0, as Linux did. `register_dccp()` registers a port's dissector, and `register_dccp_option()` an option parser. The registries are `pcapkit.const.dccp`, crawled from IANA. Its `make()` refuses a value a field cannot hold with `ProtocolError` naming the field ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2670](https://github.com/JarryShaw/PyPCAPKit/pull/2670)), and an option given as empty raw octets ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). -- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)), and a name, or a code too wide, for an enumeration field with `ProtocolError`, as the other makers do, below ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. `pcapkit.protocols.data` and `pcapkit.protocols.schema` export a `tls12_cid` record's classes as `DTLS_CIDCiphertext` and the extension's data model as `TLS_ConnectionId` ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). +- **Added** -- `TLS` (SSLv3 and TLS 1.0 to 1.3) at TCP 443 and `DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract `SSL` base ([#2437](https://github.com/JarryShaw/PyPCAPKit/issues/2437), [#2441](https://github.com/JarryShaw/PyPCAPKit/pull/2441)). Each reads every version of its record layer and reports it through `alias`, as `ARP` does by operation, telling 1.3 apart by its `supported_versions` extension; `id()` stays `('TLS',)` or `('DTLS',)`. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, compression methods and extensions, EncryptedExtensions down to its extensions, any other message at type and length -- as are an alert and ChangeCipherSpec, while encrypted records stay opaque octets. An extension's body is read and built for `server_name`, `supported_groups`, `signature_algorithms`, ALPN, `supported_versions` and `key_share`, a HelloRetryRequest's included, and kept as octets for any other type or for a body its parser rejects, which logs nothing; `register_tls_extension()` registers a parser and constructor for a further type ([#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). `make()` refuses a value too wide for its field, in the record header or in the content it builds, with `StructError` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)), and a name, or a code too wide, for an enumeration field with `ProtocolError`, as the other makers do, below ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- `TCP:TLS 1.3:TLS 1.2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2517](https://github.com/JarryShaw/PyPCAPKit/issues/2517), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). `SSLv3`, in `pcapkit.protocols.application.sslv3` ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2602](https://github.com/JarryShaw/PyPCAPKit/pull/2602)), exists for explicit construction only. A record split across TCP segments is not reassembled and stays `Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of [RFC 9147 Section 4.1](https://datatracker.ietf.org/doc/html/rfc9147#section-4.1) ([#2467](https://github.com/JarryShaw/PyPCAPKit/issues/2467), [#2494](https://github.com/JarryShaw/PyPCAPKit/pull/2494)) and [RFC 9146](https://datatracker.ietf.org/doc/html/rfc9146) `tls12_cid` records. A `DTLSContext`, given to `extract()` as its `context`, resolves the connection ID of a `tls12_cid` record, or of a unified-header record with its `C` bit set ([#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625), [#2643](https://github.com/JarryShaw/PyPCAPKit/pull/2643)), from those the capture's hellos negotiate in their `connection_id` extension, which TLS and DTLS both decode, or from the `cid_length` the caller gives it, and one it cannot resolve stays Raw. An ID a DTLS 1.3 `NewConnectionId` issues is encrypted, so only the caller can give it. A unified-header record's `info` carries its ID as `cid`, `None` when the `C` bit is clear, as `length` is when the `L` bit is, so that the one header keeps one data model, as settled on [#2625](https://github.com/JarryShaw/PyPCAPKit/issues/2625); its `make()` takes a `cid` and sets the `C` bit. A `tls12_cid` record's schema and data model are `DTLSCIDCiphertext`, in the DTLS package, and the extension's data model is `ConnectionId`, in the SSL package's ([#2515](https://github.com/JarryShaw/PyPCAPKit/issues/2515), [#2631](https://github.com/JarryShaw/PyPCAPKit/pull/2631), [#2641](https://github.com/JarryShaw/PyPCAPKit/pull/2641)). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read `DTLS 1.2:DTLS 1.3` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is `Raw` with the rest of the datagram ([#2535](https://github.com/JarryShaw/PyPCAPKit/issues/2535), [#2603](https://github.com/JarryShaw/PyPCAPKit/pull/2603)). The content, handshake and extension types are `pcapkit.const.tls`, crawled from IANA ([#2495](https://github.com/JarryShaw/PyPCAPKit/issues/2495), [#2514](https://github.com/JarryShaw/PyPCAPKit/issues/2514), [#2503](https://github.com/JarryShaw/PyPCAPKit/pull/2503), [#2595](https://github.com/JarryShaw/PyPCAPKit/pull/2595)). - **Added** -- `SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's ([#2457](https://github.com/JarryShaw/PyPCAPKit/issues/2457), [#2466](https://github.com/JarryShaw/PyPCAPKit/pull/2466)). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields agree, so anything else stays `Raw`; a kept one drops the rejected TLS trial's warnings ([#2525](https://github.com/JarryShaw/PyPCAPKit/issues/2525), [#2537](https://github.com/JarryShaw/PyPCAPKit/pull/2537)). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a `CLIENT-HELLO` then a `CLIENT-MASTER-KEY` read `SSLv2:SSLv2` -- and one that fails to decode is `Raw` with the rest of the segment ([#2567](https://github.com/JarryShaw/PyPCAPKit/issues/2567), [#2585](https://github.com/JarryShaw/PyPCAPKit/pull/2585)). Its constants are `pcapkit.const.sslv2`, from a hand-maintained table. @@ -46,7 +46,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, parsed once as `Raw`, and the walk stops there, as [RFC 8200 Section 4](https://datatracker.ietf.org/doc/html/rfc8200#section-4) has a node discard such a packet ([#2477](https://github.com/JarryShaw/PyPCAPKit/issues/2477), [#2480](https://github.com/JarryShaw/PyPCAPKit/pull/2480)). It used to fall back to the generic `IPv6_Ext` and walk on; `IPv6_Ext` now parses only Shim6, which has no dedicated parser, and `IPv6.__generic_ext_codes__`, the headers that fell back, is removed. The library no longer passes `IPv6_Ext.read()` an `error=`, so `IPv6_Ext.error` is `None` unless a caller supplies one; the `Raw` layer's `info.error` holds the failure instead. -- **Changed** -- **a breaking change to** parsing a CALIPSO, SMF_DPD or MPL option in `HOPOPT` or `IPv6_Opts`: an Opt Data Len too short for the option raises `ProtocolError`, where 1.5.x raised `FieldValueError`, so code catching `FieldValueError` for it catches `ProtocolError` instead, both being `ValueError`; inside IPv6 the header is kept as `Raw` with nothing logged, where 1.5.x logged it at `CRITICAL` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). So do the length callbacks of `pcapkit.protocols.schema.internet.ipv6_option` that size those options -- `calipso_pad_len`, `mpl_opt_pad_len`, `smf_i_dpd_id_len` and `smf_dpd_data_selector`. A capture cut after such an option's type octet reads as a short read, a CALIPSO or SMF_DPD option kept and rebuilt as captured and an MPL option refused quietly, where 1.5.x raised `FieldValueError: HOPOPT: invalid CALIPSO option length: 0` logged at `CRITICAL`. +- **Changed** -- **a breaking change to** parsing a CALIPSO, SMF_DPD or MPL option in `HOPOPT` or `IPv6_Opts`: an Opt Data Len too short for the option raises `ProtocolError`, where 1.5.x raised `FieldValueError`, so code catching `FieldValueError` for it catches `ProtocolError` instead, both being `ValueError`; inside IPv6 the header is kept as `Raw` with nothing logged, where 1.5.x logged it at `CRITICAL` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). So do the length callbacks of `pcapkit.protocols.internet.ipv6_ext.schema` that size those options -- `calipso_pad_len`, `mpl_opt_pad_len`, `smf_i_dpd_id_len` and `smf_dpd_data_selector`. A capture cut after such an option's type octet reads as a short read, a CALIPSO or SMF_DPD option kept and rebuilt as captured and an MPL option refused quietly, where 1.5.x raised `FieldValueError: HOPOPT: invalid CALIPSO option length: 0` logged at `CRITICAL`. - **Changed** -- **a breaking change to** parsing a Mobility Header or a Multipath TCP DSS option whose length does not fit its fields: it raises `ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured ([#2669](https://github.com/JarryShaw/PyPCAPKit/issues/2669), [#2683](https://github.com/JarryShaw/PyPCAPKit/pull/2683)). `MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message [RFC 6275 Section 9.2](https://datatracker.ietf.org/doc/html/rfc6275#section-9.2) has discarded, so a Home Test with Header Len 1 raises `MH: [Type 3] invalid header length: 1`, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as `Raw` by the walk above, and the packet rebuilds as captured, where its 64 octets rebuilt as 72. `TCP` refuses a DSS Length other than the one [RFC 8684](https://datatracker.ietf.org/doc/html/rfc8684) gives its `A`, `a`, `M` and `m` flags, or that plus the 2-octet checksum with `M` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises `TCP: [OptNo 30] invalid format`, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as `Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. A mobility option whose fields do not take its Length is refused too, and so are such a sub-option, QoS attribute or CGA extension inside one, a target care-of address neither IPv4 nor IPv6, and CGA Parameters whose fixed fields or public key run past the option, where 1.5.x read octets that were not the option's and rebuilt another ([#2684](https://github.com/JarryShaw/PyPCAPKit/issues/2684), [#2700](https://github.com/JarryShaw/PyPCAPKit/pull/2700)). So an MN-ID option of Length 0 raises `MH: [OptNo 8] invalid option length: 0`, where 1.5.x read its fields from the octets after it and rebuilt it with Length 17, and a Flow Identification option's 8-octet target care-of address raises `MH: [FID SubOpt 5] invalid sub-option length: 10`, where 1.5.x parsed it and raised `FieldValueError` rebuilding it; inside IPv6 the header is `Raw` and the packet rebuilds as captured. @@ -64,13 +64,15 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- **a breaking change to** construction: a `make()` field holding an enumeration takes the member or an `int` code, resolved with the enumeration's `get`, and the `X_default`, `X_namespace` and `X_reversed` keywords beside such a field are removed with no alias ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). The makers of IPv4, IPv6 and its extension headers, IPX, TCP, ARP, Ethernet, VLAN, L2TPv2, OSPF, HTTP, the PCAP header and PCAP-NG took them in 1.5.x, and construction given one raises `UnsupportedCall`, as `IPv4: unexpected keyword(s): 'protocol_default' (did you mean 'protocol'?)`. A name raises `ProtocolError`, as `IPv4: invalid protocol: 'TCP'`, where 1.5.x resolved it against the field's `X_namespace` or an enumeration it inferred from the caller's frame. So does a name given as NGAP's `procedure` or `criticality`, which 1.5.x resolved to its member. A caller holding a name resolves it first with `Protocol.resolve_index`, given the namespace, a default and whether a `dict` namespace maps names to codes. A code is checked against the field's width as an integer field's is, above, so `IPv4(protocol=256)` raises `IPv4: invalid protocol: 256`, where it wrote `0x00`, and `IPv4(protocol=True)` raises, where it wrote `0x01`. So is the code of an option, sub-option or chunk given as a `(code, arguments)` pair: MH's option type 300 raises `MH: invalid option type: 300` and TCP's option kind 300 `TCP: invalid option kind: 300`, where 1.5.x wrote `0x2C` ([#2693](https://github.com/JarryShaw/PyPCAPKit/issues/2693)). An `int` HTTP/1 status given without a `message` writes its registry phrase, as a member did, so `404` writes `404 Not Found`, `299` `299 Unassigned` and `600` `600 Unknown`, where 1.5.x wrote the code with an empty phrase. -- **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a module holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. +- **Changed** -- **a breaking change to** `DRARP` and `InARP`, which move from `pcapkit.protocols.application.rarp` to `pcapkit.protocols.application.drarp` and from `pcapkit.protocols.link.arp` to `pcapkit.protocols.link.inarp`, with no re-export at the old paths, as a package holds one protocol class ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2540](https://github.com/JarryShaw/PyPCAPKit/pull/2540), [#2596](https://github.com/JarryShaw/PyPCAPKit/pull/2596)). `pcapkit.DRARP`, `pcapkit.InARP`, their `pcapkit.protocols` exports and a parsed RARP's `DRARP` or ARP's `InARP` alias are unchanged. -- **Changed** -- **a breaking change to** `NoPayload`, which moves from `pcapkit.protocols.misc.null` to `pcapkit.protocols.misc.no_payload`, and its data model and header schema from `pcapkit.protocols.data.misc.null` and `pcapkit.protocols.schema.misc.null` to `pcapkit.protocols.data.misc.no_payload` and `pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old paths, as a protocol class takes its module's name ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2636](https://github.com/JarryShaw/PyPCAPKit/pull/2636)). The class keeps its name, and `pcapkit.NoPayload` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails to load, as the pickle names the old module, though its pickled `info` loads. +- **Changed** -- **a breaking change to** `NoPayload`, which moves from `pcapkit.protocols.misc.null` to `pcapkit.protocols.misc.no_payload`, and its data model and header schema from `pcapkit.protocols.data.misc.null` and `pcapkit.protocols.schema.misc.null` to that package's `data` and `schema`, with no re-export at the old paths, as a protocol class takes its package's name ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2636](https://github.com/JarryShaw/PyPCAPKit/pull/2636)). The class keeps its name, and `pcapkit.NoPayload` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails to load, as any 1.5.x pickle of a data model does, below. + +- **Changed** -- **a breaking change to** the layout of `pcapkit.protocols`: each protocol is a package holding its dissector, header schema and data model as `protocol`, `schema` and `data` modules, as `pcapkit.protocols.internet.ipv4` holds `protocol`, `schema` and `data`, where 1.5.x kept the schemas and data models in the separate `pcapkit.protocols.schema` and `pcapkit.protocols.data` trees ([#2534](https://github.com/JarryShaw/PyPCAPKit/issues/2534), [#2753](https://github.com/JarryShaw/PyPCAPKit/pull/2753)). Those trees and their aggregate exports are removed with no alias, so `import pcapkit.protocols.data.internet.ipv4` and `from pcapkit.protocols.data import PCAPNG_IF_NameOption` raise `ImportError`. A package's `__init__.py` exports its protocol class and binds each schema and data class as `Schema_` and `Data_`, so `from pcapkit.protocols.internet.ipv4 import IPv4, Data_IPv4, Schema_IPv4` reads as before, and `pcapkit.protocols.misc.pcapng.Data_IF_NameOption` replaces the aggregate `PCAPNG_IF_NameOption`. The other names of a 1.5.x protocol module are in its package's `protocol` module, and a `versionremoved` note in the API reference gives each one that moved elsewhere. `pcapkit.protocols.schema` and `pcapkit.protocols.data` are modules holding the base classes, `Schema` and `Data` among them, and the IPv6 option helpers of `pcapkit.protocols.schema.internet.ipv6_option` are in `pcapkit.protocols.internet.ipv6_ext.schema`. `pcapkit.IPv4` and the `pcapkit.protocols` exports are unchanged. A data model pickled by 1.5.x fails to load, as the pickle names its 1.5.x module, raising `ModuleNotFoundError: No module named 'pcapkit.protocols.data.internet'` say. - **Changed** -- **a breaking change to** the schema helpers that code outside the schema layer calls, which move to `pcapkit.corekit.schema`, a package with a module per group, with no re-export at their old paths ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). `parse_ip_address()` moves from `pcapkit.corekit.fields.ipaddress` to `pcapkit.corekit.schema.ipaddress`; `packet_byteorder()` and `byteorder_callback()` from `pcapkit.corekit.fields.byteorder`, which is removed with no alias, to `pcapkit.corekit.schema.byteorder`; `jumbo_payload_length()` from `pcapkit.protocols.schema.internet.ipv6` to `pcapkit.corekit.schema.jumbogram`; and `keep_short_read()` and `replay_short_read()` from `pcapkit.protocols.schema.schema` to `pcapkit.corekit.schema.shortread`. The header schema framework stays `pcapkit.protocols.schema`. -- **Changed** -- **a breaking change to** the mapping values a caller reads as a `dict`: each is an `Info` or a data model, not a `dict` ([#2726](https://github.com/JarryShaw/PyPCAPKit/issues/2726), [#2734](https://github.com/JarryShaw/PyPCAPKit/pull/2734)). `FrameRecord`, the frame a third-party engine's flow tracing reports, is an `Info` where 1.5.x subclassed `dict`, so `isinstance(record, dict)` is false, setting a key raises `TypeError`, and `to_dict()` exports the engine's mapping as an `OrderedMultiDict` without the `packet`, `frame_info` and `time_epoch` attributes, which stay attributes, as in 1.5.x. A parsed TLS key log's `TLSKeyLog.entries` is a `TLSKeyLogEntries` keyed by label, which `pcapkit.protocols.data` exports as `PCAPNG_TLSKeyLogEntries`, where it was a `dict`; a mapping given for it is stored as one, and `to_dict()` exports it as an `OrderedMultiDict`, where it gave a `dict`. Pickles of either load across 1.5.x and 1.6.0 in both directions. `AppType.__registries__` is an `Info` keyed by `TransportProtocol`, where 1.5.x held a `dict` subclass, so setting a key raises `TypeError` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). +- **Changed** -- **a breaking change to** the mapping values a caller reads as a `dict`: each is an `Info` or a data model, not a `dict` ([#2726](https://github.com/JarryShaw/PyPCAPKit/issues/2726), [#2734](https://github.com/JarryShaw/PyPCAPKit/pull/2734)). `FrameRecord`, the frame a third-party engine's flow tracing reports, is an `Info` where 1.5.x subclassed `dict`, so `isinstance(record, dict)` is false, setting a key raises `TypeError`, and `to_dict()` exports the engine's mapping as an `OrderedMultiDict` without the `packet`, `frame_info` and `time_epoch` attributes, which stay attributes, as in 1.5.x. A parsed TLS key log's `TLSKeyLog.entries` is a `TLSKeyLogEntries` keyed by label, which `pcapkit.protocols.misc.pcapng` binds as `Data_TLSKeyLogEntries`, where it was a `dict`; a mapping given for it is stored as one, and `to_dict()` exports it as an `OrderedMultiDict`, where it gave a `dict`. `AppType.__registries__` is an `Info` keyed by `TransportProtocol`, where 1.5.x held a `dict` subclass, so setting a key raises `TypeError` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). - **Changed** -- **a breaking change to** HTTP: parsing octets through `HTTP`, building it from fields and its `from_data` give the `HTTPv1` or `HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under `httpv1` or `httpv2` where it sat under `http` ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)). Building it from fields takes `version`, `1` or `2`, as the other versioned bases above do, and raises `ProtocolError` without it, where 1.5.x built an HTTP/1.1 message; its `from_data` takes the member from the data model's class and refuses a `dict`, where 1.5.x took one ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The members are renamed after their modules, `httpv1.HTTP` and `httpv2.HTTP` becoming `HTTPv1` and `HTTPv2` along with their schema and data classes, with no re-export at the old paths ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529)); `pcapkit.HTTPv1`, `pcapkit.HTTPv2` and the package exports of those names are unchanged. `id()` takes the RFC names, `('HTTP/1',)` and `('HTTP/2',)`, and the base's lists both, where they were `('HTTP', 'HTTPv1')`, `('HTTP', 'HTTPv2')` and `('HTTP', 'HTTPv1', 'HTTPv2')`. So a layer no longer compares equal to `'HTTP'`, `'HTTPv1'` or `'HTTPv2'`; `'HTTP'` still selects either version, and each version's own name, `'HTTP/1'` or `'HTTPv1'` say, selects it alone, where any HTTP layer matched `'HTTPv1'` and `'HTTPv2'` both. On the transport path a payload neither version recognises, a bare connection preface included, is `Raw` with no `error`, where it carried `'HTTP: unknown version'` or `'HTTP/2: connection preface with no frame'`; a direct `HTTP(...)` on it still raises `ProtocolError`. One that passes for an HTTP/2 frame header but `HTTPv2` refuses carries that parser's own error on either path, where it was `'HTTP: unknown version'`. `HTTPv2` itself now parses a payload opening with the connection preface, kept as `preface`, and its `make()` takes `preface`, which `HTTPv1` refuses, where 1.5.x wrote the HTTP/2 preface ahead of the HTTP/1 message ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). @@ -90,13 +92,13 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its module's name, one to a module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). A next-layer refusal in a payload a layer rebuilds is quiet, as the layer's own parse reports it once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). +- **Changed** -- the house conventions record that a protocol class takes its package's name, one to a package ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). A next-layer refusal in a payload a layer rebuilds is quiet, as the layer's own parse reports it once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). A protocol is a package of `protocol`, `schema` and `data` modules, whose `__init__.py` exports its protocol class and binds its schema and data classes as `Schema_*` and `Data_*` outside `__all__`, which a `[tool.isort]` section keeps isort off ([#2534](https://github.com/JarryShaw/PyPCAPKit/issues/2534), [#2753](https://github.com/JarryShaw/PyPCAPKit/pull/2753)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). - **Changed** -- the `docs` extra pins `sphinx-autodoc-typehints` 3.17.4, where 1.5.x pinned 3.17.2 ([#2742](https://github.com/JarryShaw/PyPCAPKit/pull/2742)). -- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). The GraalPy leg there splits the long shards whose xdist workers outgrew their heap and died, each sized from its tests' JUnit seconds to run in under 5 minutes ([#2713](https://github.com/JarryShaw/PyPCAPKit/issues/2713), [#2720](https://github.com/JarryShaw/PyPCAPKit/pull/2720)). Its tests' timeout is 180 minutes, where it was 110, and the internet and application protocol shards are split too, as a slower runner ran out of time with no worker lost ([#2739](https://github.com/JarryShaw/PyPCAPKit/issues/2739), [#2741](https://github.com/JarryShaw/PyPCAPKit/pull/2741)). +- **Changed** -- the CPython unit-test legs run `pytest -n logical`, a worker per logical CPU, where `-n auto` counted physical cores and ran 2 workers on the 4-CPU runner; the weekly CI timing report reads the latest Python's leg too ([#2571](https://github.com/JarryShaw/PyPCAPKit/issues/2571), [#2572](https://github.com/JarryShaw/PyPCAPKit/pull/2572)). The integration, engine, parity, project-test and gate jobs and the experimental workflow's Windows and free-threaded 3.14t legs run `-n logical` too, and a project test fails on a bare `-n auto` ([#2573](https://github.com/JarryShaw/PyPCAPKit/issues/2573), [#2606](https://github.com/JarryShaw/PyPCAPKit/pull/2606)). A PyPy leg runs `tests/protocols/internet`, `tests/protocols/application` and `tests/protocols/misc` as shards of their own ahead of the rest of `tests/protocols`, which run whole outgrew the runner's memory and stalled at its tail ([#2600](https://github.com/JarryShaw/PyPCAPKit/issues/2600), [#2604](https://github.com/JarryShaw/PyPCAPKit/pull/2604)). The experimental workflow's Windows leg passes pytest the modules it ignores as an `@`-prefixed argument file, where they went on its command line and had outgrown Windows' limit, so the leg failed before pytest started ([#2678](https://github.com/JarryShaw/PyPCAPKit/issues/2678), [#2681](https://github.com/JarryShaw/PyPCAPKit/pull/2681)). The GraalPy leg there splits the long shards whose xdist workers outgrew their heap and died, each sized from its tests' JUnit seconds to run in under 5 minutes ([#2713](https://github.com/JarryShaw/PyPCAPKit/issues/2713), [#2720](https://github.com/JarryShaw/PyPCAPKit/pull/2720)). Its tests' timeout is 180 minutes, where it was 110, and the internet and application protocol shards are split too, as a slower runner ran out of time with no worker lost ([#2739](https://github.com/JarryShaw/PyPCAPKit/issues/2739), [#2741](https://github.com/JarryShaw/PyPCAPKit/pull/2741)). The `latest` leg's `LATEST` test tier runs in a job of its own, in halves, and a `Coverage` job combines the coverage legs' parts, posting no total unless every part arrived ([#2751](https://github.com/JarryShaw/PyPCAPKit/issues/2751), [#2754](https://github.com/JarryShaw/PyPCAPKit/pull/2754)). The plain unittest ordering leg runs the rest of `tests/protocols` in two halves, as the cell had reached its 20-minute step cap with no test failed ([#2751](https://github.com/JarryShaw/PyPCAPKit/issues/2751)). - **Changed** -- once a release's milestone is closed while `__version__` still names that release, the milestone workflow gives an item the lowest open release milestone above it, compared as numbers so `v1.10` follows `v1.9`, and a backfill starts above the closed milestone's items ([#2548](https://github.com/JarryShaw/PyPCAPKit/issues/2548), [#2591](https://github.com/JarryShaw/PyPCAPKit/pull/2591)). @@ -128,7 +130,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- `TCP`'s maker, given an option as raw octets, raised `EnumValueError` for a kind its registry does not name, such as 255, and `IndexError` for empty octets. The kind resolves as a `(code, arguments)` pair's does, so 255 builds, and empty octets raise `ProtocolError` ([#2708](https://github.com/JarryShaw/PyPCAPKit/issues/2708), [#2724](https://github.com/JarryShaw/PyPCAPKit/pull/2724)). The other makers taking a raw item refuse empty octets the same way, as `IPv4: invalid option: b'' holds no type octet` ([#2723](https://github.com/JarryShaw/PyPCAPKit/issues/2723), [#2728](https://github.com/JarryShaw/PyPCAPKit/pull/2728)). IPv4's, HOPOPT's and IPv6-Opts' options raised `IndexError`. SCTP's parameters and error causes, and MH's options, sub-options, QoS attributes and CGA extensions, raised a bare `AttributeError` alone -- or, for MH's access network identifier and LMA-controlled MAG sub-options, a `ProtocolError` for an option of length 0 -- and were dropped beside a whole item. HIP's parameters were dropped. PCAP-NG's options and Name Resolution records refuse octets shorter than their 2-octet code, where 1.5.x wrote empty or zero octets as an end-of-options or end-of-records marker, and misread one other octet as a code. `HIP(next=59)` wrote HIP version 6, taking the IP version for it, and writes 2, `make()`'s default from [RFC 7401 Section 5.1](https://datatracker.ietf.org/doc/html/rfc7401#section-5.1). HIP's `ENCRYPTED` maker refuses an IV given with a cipher that takes none, which 1.5.x counted in the parameter's Length but did not write. A parsed `ENCRYPTED` parameter takes its cipher from the `HIP_CIPHER` before it, where 1.5.x warned `HIP: [ParamNo 641] missing HIP_CIPHER parameter` and read an AES IV as encrypted data. MH's CGA Parameters maker builds a raw extension in a `CGAParameter` data model as its `(type, arguments)` pair builds, where 1.5.x raised `AttributeError: 'bytes' object has no attribute 'type'`, and a `CGAParameter` schema without `public_key_test`, leaving the schema given unchanged, where 1.5.x raised `KeyError: 'public_key_test'` ([#2725](https://github.com/JarryShaw/PyPCAPKit/issues/2725), [#2732](https://github.com/JarryShaw/PyPCAPKit/pull/2732)). -- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.schema.misc.pcapng.zigbee_key_held` returns a schema whose `bytes` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). +- **Fixed** -- a `PCAPNG` Decryption Secrets Block that `make()` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and `pcapkit.protocols.misc.pcapng.schema.zigbee_key_held` returns a schema whose `bytes` and `len()` are those of the octets held, where they were the fixed width's ([#2688](https://github.com/JarryShaw/PyPCAPKit/issues/2688), [#2697](https://github.com/JarryShaw/PyPCAPKit/pull/2697)). Rebuilt by `from_data` from the block's `dict`, a TLS or WireGuard key log was written from its entries under a `# generated by PyPCAPKit` line, dropping its comments and the lines its entries do not hold, and a ZigBee key cut short of its fixed width was written at that width, zero-filled. The TLS, WireGuard and ZigBee secrets makers take the captured text or octets as `data`, so the block rebuilds as captured from its `dict`, as from its data model ([#2698](https://github.com/JarryShaw/PyPCAPKit/issues/2698), [#2707](https://github.com/JarryShaw/PyPCAPKit/pull/2707)). - **Fixed** -- rebuilding a layer through `from_data` reported its payload's messages twice, once from the payload's own rebuild and again from the layer's parse of it: in 1.5.x, `Ethernet.from_data` of a frame cut inside its TCP header logged the refusal twice, where the parse logged it once. Each is reported once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). IPv4 records a cut inside the payload length it declares as its `__short_read__`, where 1.5.x recorded none, and so do Ethernet, GRE, ICMP, IGMP and a TSO IPv4 layer for a cut at the end of their frame. diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 8d88dd021f..6d21352fe7 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -9,11 +9,11 @@ pseudo-header, verified and computed one's-complement checksums, stricter handling of a malformed IPv6 extension header, and HTTP dispatched to its HTTP/1 and HTTP/2 classes under the RFC names. -* **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.LinuxSLL1` and - :class:`~pcapkit.protocols.link.linux_sll2.LinuxSLL2`, dissectors for the +* **Added** -- :class:`~pcapkit.protocols.link.linux_sll1.protocol.LinuxSLL1` and + :class:`~pcapkit.protocols.link.linux_sll2.protocol.LinuxSLL2`, dissectors for the Linux "cooked" capture headers ``tcpdump -i any`` writes, under ``LINKTYPE_LINUX_SLL`` (113) and ``LINKTYPE_LINUX_SLL2`` (276), on a - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` family base; PCAP and + :class:`~pcapkit.protocols.link.linux_sll.protocol.LinuxSLL` family base; PCAP and PCAP-NG frames of either link type dispatch to them (:issue:`2438`, :pr:`2443`). ``'LinuxSLL'`` selects either version, and each version's own name selects that version alone. ``protocol_type`` dispatches onward as an @@ -26,7 +26,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``code=ARPHRD.X``, since ``ARPHRD`` names ``LinuxSLL`` as its ``__proto_dest__`` (:issue:`2533`, :pr:`2598`). A parse told to stop there -- at ``LinuxSLL``, the version itself or the link layer -- keeps the payload - :class:`~pcapkit.protocols.misc.raw.Raw` on either path (:issue:`2532`, + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` on either path (:issue:`2532`, :pr:`2587`). The hardware types are :class:`~pcapkit.const.linux.arphrd.ARPHRD`, generated by :class:`pcapkit.vendor.linux.arphrd.ARPHRD` from a hand-maintained table. It @@ -35,14 +35,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. :pr:`2497`), and :mod:`pcapkit.const` exports it as ``Linux_ARPHRD`` (:issue:`2661`, :pr:`2663`). -* **Added** -- :class:`~pcapkit.protocols.link.frad.FRAD`, a dissector for Frame +* **Added** -- :class:`~pcapkit.protocols.link.frad.protocol.FRAD`, a dissector for Frame Relay's multiprotocol encapsulation (:rfc:`2427`, obsoleting :rfc:`1490`), - which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for + which :class:`~pcapkit.protocols.link.linux_sll.protocol.LinuxSLL` dispatches to for ``ARPHRD_FRAD`` (:pr:`2499`), as do PCAP and PCAP-NG frames of ``LINKTYPE_FRELAY`` (107) (:issue:`2649`, :pr:`2667`). The direct NLPID and SNAP-encapsulated forms both dispatch onward, to IPv4/IPv6 and (via the EtherType registry every Ethernet/VLAN/LinuxSLL frame shares) ARP among others; any other NLPID or - SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, and + SNAP OUI is kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`, and :func:`~pcapkit.foundation.registry.protocols.link.register_frad` registers a dissector for a further NLPID (:pr:`2527`). A Q.922 address other than the 2-octet form, its EA bits other than 0 then 1, is refused with @@ -56,24 +56,24 @@ HTTP/1 and HTTP/2 classes under the RFC names. by default writing one ahead of a SNAP header or an NLPID of 0, and refuses ``pad=False`` with an NLPID of 0 (:issue:`2716`, :pr:`2721`). -* **Added** -- :class:`~pcapkit.protocols.link.netlink.Netlink`, a dissector for +* **Added** -- :class:`~pcapkit.protocols.link.netlink.protocol.Netlink`, a dissector for the fixed 16-octet ``nlmsghdr`` framing of Linux's kernel/userspace IPC transport (the kernel's ``netlink(7)`` man page; :rfc:`3549#section-2.3.2`), which - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for + :class:`~pcapkit.protocols.link.linux_sll.protocol.LinuxSLL` dispatches to for ``ARPHRD_NETLINK`` (:issue:`2500`, :pr:`2507`). Netlink is IPC rather than a wire format, so the header's byte order is ambiguous; - :func:`~pcapkit.protocols.link.netlink.nlmsg_byteorder` guesses it the way + :func:`~pcapkit.protocols.link.netlink.protocol.nlmsg_byteorder` guesses it the way Wireshark's netlink dissector does. Only the first message's header is dissected: what follows it, its per-family payload and any further message - in the same capture, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` + in the same capture, is kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw` unless :func:`~pcapkit.foundation.registry.protocols.link.register_netlink` registers a dissector for its ``nlmsg_type`` (:pr:`2527`). -* **Added** -- :class:`~pcapkit.protocols.link.gre.GRE`, a dissector for the +* **Added** -- :class:`~pcapkit.protocols.link.gre.protocol.GRE`, a dissector for the Generic Routing Encapsulation base header (:rfc:`2784`) plus the optional Key and Sequence Number extensions (:rfc:`2890`), which - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` dispatches to for both + :class:`~pcapkit.protocols.link.linux_sll.protocol.LinuxSLL` dispatches to for both ``ARPHRD_IPGRE`` and ``ARPHRD_IP6GRE`` -- the same GRE framing either way, the two differing only in the tunnel's outer IP version (:issue:`2501`, :pr:`2508`), as does an IP layer for IP protocol 47, through the registry @@ -85,12 +85,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. through the family registry (:attr:`Link.__proto__ `) Ethernet, VLAN and LinuxSLL share. A non-zero Version, :rfc:`2637`'s - Enhanced GRE, is kept as :class:`~pcapkit.protocols.misc.raw.Raw` rather than + Enhanced GRE, is kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw` rather than misread. -* **Added** -- :class:`~pcapkit.protocols.link.radiotap.Radiotap`, a dissector +* **Added** -- :class:`~pcapkit.protocols.link.radiotap.protocol.Radiotap`, a dissector for the radiotap header () capture tools prepend to - an 802.11 frame, which :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` + an 802.11 frame, which :class:`~pcapkit.protocols.link.linux_sll.protocol.LinuxSLL` dispatches to for ``ARPHRD_IEEE80211_RADIOTAP`` (:issue:`2502`, :pr:`2509`), as do PCAP and PCAP-NG frames of ``LINKTYPE_IEEE802_11_RADIOTAP`` (127) (:issue:`2608`, :pr:`2632`). It decodes the fixed 8-octet start (``it_version``, ``it_pad``, ``it_len``) @@ -106,17 +106,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. cut short of its layout, as radiotap.org allows, decodes with the octets it lacks as zeros and rebuilds as captured (:issue:`2655`, :pr:`2690`). A bit no field is defined for, or a field running past the region, ends the walk - as an :class:`~pcapkit.protocols.data.link.radiotap.Unknown` entry holding + as an :class:`~pcapkit.protocols.link.radiotap.data.Unknown` entry holding the rest, and ``make()`` takes the fields decoded as well as raw. Each field type has a concrete - :class:`~pcapkit.protocols.schema.link.radiotap.RadiotapField` schema, and + :class:`~pcapkit.protocols.link.radiotap.schema.RadiotapField` schema, and is read and made through - :attr:`Radiotap.__field__ `, + :attr:`Radiotap.__field__ `, which - :meth:`Radiotap.register_field ` + :meth:`Radiotap.register_field ` and :func:`~pcapkit.foundation.registry.protocols.link.register_radiotap_field` - extend; :mod:`pcapkit.protocols.schema` and :mod:`pcapkit.protocols.data` - export the schemas and data models with a ``Radiotap_`` prefix. A bitmap + extend. A bitmap radiotap.org defines bit by bit is a member of its flag registry, which keeps the bits it does not name. One mixing value subfields with flags -- MCS ``flags``, HE ``data1`` and ``data2``, HE-MU ``flags1`` and ``flags2`` @@ -134,32 +133,32 @@ HTTP/1 and HTTP/2 classes under the RFC names. takes the name of its macro in Linux's ``ieee80211_radiotap.h``, or in Wireshark where Linux defines none, and radiotap.org's wording where neither does. The 802.11 MAC frame behind the header is dissected as - :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, below, looked up in + :class:`~pcapkit.protocols.link.ieee802_11.protocol.IEEE802_11`, below, looked up in ``Radiotap``'s own registry under :attr:`LinkType.IEEE802_11 `, and :func:`~pcapkit.foundation.registry.protocols.link.register_radiotap` replaces that dissector (:issue:`2575`, :pr:`2607`). -* **Added** -- :class:`~pcapkit.protocols.link.ieee802_11.IEEE802_11`, a +* **Added** -- :class:`~pcapkit.protocols.link.ieee802_11.protocol.IEEE802_11`, a dissector for the IEEE 802.11 MAC header of a protocol version 0 frame, - which :class:`~pcapkit.protocols.link.radiotap.Radiotap` dispatches to + which :class:`~pcapkit.protocols.link.radiotap.protocol.Radiotap` dispatches to (:issue:`2575`, :pr:`2607`), as do PCAP and PCAP-NG frames of ``LINKTYPE_IEEE802_11`` (105) (:issue:`2608`, :pr:`2632`). It reads Frame Control, the Control Frame Extension and S1G Beacon layouts included, and the address, sequence, QoS Control and HT Control fields each frame type carries; a Control frame of a subtype it does not model, and an Extension frame, are read as the minimal - header. The frame body is kept as :class:`~pcapkit.protocols.misc.raw.Raw`, + header. The frame body is kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`, with the FCS and radiotap's pad octets where captured, as ``Radiotap`` decodes the Flags that mark them but does not pass them on, and any other protocol version is refused. The frame types and subtypes are :mod:`pcapkit.const.ieee802_11`, from hand-maintained tables of IEEE Std 802.11-2020's Table 9-1. -* **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.ICMPv4` (:rfc:`792`) - and :class:`~pcapkit.protocols.internet.icmpv6.ICMPv6` (:rfc:`4443`), on an - abstract :class:`~pcapkit.protocols.internet.icmp.ICMP` base, for IP +* **Added** -- :class:`~pcapkit.protocols.internet.icmpv4.protocol.ICMPv4` (:rfc:`792`) + and :class:`~pcapkit.protocols.internet.icmpv6.protocol.ICMPv6` (:rfc:`4443`), on an + abstract :class:`~pcapkit.protocols.internet.icmp.protocol.ICMP` base, for IP protocols 1 and 58, which were kept as - :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2436`, :pr:`2446`); a + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` (:issue:`2436`, :pr:`2446`); a protochain names ``ICMPv6`` where it named ``IPv6_ICMP``. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD @@ -170,11 +169,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. :mod:`pcapkit.const.icmp` and :mod:`pcapkit.const.icmpv6`, crawled from IANA. * **Added** -- IGMPv1 (:rfc:`1112`), IGMPv2 (:rfc:`2236`) and IGMPv3 - (:rfc:`3376`), as :class:`~pcapkit.protocols.internet.igmpv1.IGMPv1`, - :class:`~pcapkit.protocols.internet.igmpv2.IGMPv2` and - :class:`~pcapkit.protocols.internet.igmpv3.IGMPv3`, each named after its + (:rfc:`3376`), as :class:`~pcapkit.protocols.internet.igmpv1.protocol.IGMPv1`, + :class:`~pcapkit.protocols.internet.igmpv2.protocol.IGMPv2` and + :class:`~pcapkit.protocols.internet.igmpv3.protocol.IGMPv3`, each named after its module (:issue:`2529`, :pr:`2542`), on an abstract - :class:`~pcapkit.protocols.internet.igmp.IGMP` base bound at IP protocol 2, + :class:`~pcapkit.protocols.internet.igmp.protocol.IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 (:issue:`2439`, :issue:`2448`, @@ -193,7 +192,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. records captured and none made up, and a count no IP payload could carry raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2711`, :pr:`2738`). Any other message, a 9- to - 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.Raw`. + 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and ``('IGMPv3',)``; the base's is ``('IGMP', 'IGMPv2', 'IGMPv3')``, each @@ -205,11 +204,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. types from a hand-maintained table. * **Added** -- construction through a versioned family's base: - :class:`~pcapkit.protocols.internet.ip.IP` and - :class:`~pcapkit.protocols.link.l2tp.L2TP`, which raised :exc:`TypeError` in - 1.5.x, and the :class:`~pcapkit.protocols.internet.icmp.ICMP`, - :class:`~pcapkit.protocols.internet.igmp.IGMP` and - :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL` bases above give a member, + :class:`~pcapkit.protocols.internet.ip.protocol.IP` and + :class:`~pcapkit.protocols.link.l2tp.protocol.L2TP`, which raised :exc:`TypeError` in + 1.5.x, and the :class:`~pcapkit.protocols.internet.icmp.protocol.ICMP`, + :class:`~pcapkit.protocols.internet.igmp.protocol.IGMP` and + :class:`~pcapkit.protocols.link.linux_sll.protocol.LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and ``from_data`` (:issue:`2574`, :pr:`2639`). The ``version`` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they @@ -226,9 +225,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. version raise ``[IPv4] invalid version: 6`` say, where IPv4's error named whichever field they broke first. -* **Added** -- :class:`~pcapkit.protocols.transport.dccp.DCCP` (:rfc:`4340`) +* **Added** -- :class:`~pcapkit.protocols.transport.dccp.protocol.DCCP` (:rfc:`4340`) for IP protocol 33, which was kept as - :class:`~pcapkit.protocols.misc.raw.Raw` (:issue:`2489`, :pr:`2593`). It + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` (:issue:`2489`, :pr:`2593`). It reads and builds the generic header with short and long sequence numbers, the fields of every packet type, DCCP-Listen (:rfc:`5596`) and the options; the Data Checksum option's CRC-32c is recorded rather than checked. Its @@ -246,12 +245,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. (:issue:`2614`, :pr:`2670`), and an option given as empty raw octets (:issue:`2723`, :pr:`2728`). -* **Added** -- :class:`~pcapkit.protocols.application.tls.TLS` (SSLv3 and TLS - 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.DTLS` +* **Added** -- :class:`~pcapkit.protocols.application.tls.protocol.TLS` (SSLv3 and TLS + 1.0 to 1.3) at TCP 443 and :class:`~pcapkit.protocols.application.dtls.protocol.DTLS` (DTLS 1.0, 1.2 and 1.3) at UDP 5684, on an abstract - :class:`~pcapkit.protocols.application.ssl.SSL` base (:issue:`2437`, + :class:`~pcapkit.protocols.application.ssl.protocol.SSL` base (:issue:`2437`, :pr:`2441`). Each reads every version of its record layer and reports it - through ``alias``, as :class:`~pcapkit.protocols.link.arp.ARP` does by + through ``alias``, as :class:`~pcapkit.protocols.link.arp.protocol.ARP` does by operation, telling 1.3 apart by its ``supported_versions`` extension; ``id()`` stays ``('TLS',)`` or ``('DTLS',)``. A cleartext handshake record is decoded -- ClientHello and ServerHello down to their cipher suites, @@ -273,16 +272,16 @@ HTTP/1 and HTTP/2 classes under the RFC names. Each whole TLS record in a TCP segment is a layer of its own, the previous record's payload, so each alias names its own version -- ``TCP:TLS 1.3:TLS 1.2`` -- and one that fails to decode is - :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` with the rest of the segment (:issue:`2517`, :pr:`2537`). - :class:`~pcapkit.protocols.application.sslv3.SSLv3`, in + :class:`~pcapkit.protocols.application.sslv3.protocol.SSLv3`, in :mod:`pcapkit.protocols.application.sslv3` (:issue:`2529`, :pr:`2602`), exists for explicit construction only. A record split across TCP segments is - not reassembled and stays :class:`~pcapkit.protocols.misc.raw.Raw`. DTLS also + not reassembled and stays :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. DTLS also reads and builds the DTLS 1.3 unified-header records of :rfc:`9147#section-4.1` (:issue:`2467`, :pr:`2494`) and :rfc:`9146` ``tls12_cid`` records. A - :class:`~pcapkit.protocols.application.dtls.DTLSContext`, given to + :class:`~pcapkit.protocols.application.dtls.protocol.DTLSContext`, given to ``extract()`` as its ``context``, resolves the connection ID of a ``tls12_cid`` record, or of a unified-header record with its ``C`` bit set (:issue:`2625`, :pr:`2643`), from those the capture's hellos negotiate in @@ -293,43 +292,43 @@ HTTP/1 and HTTP/2 classes under the RFC names. :data:`None` when the ``C`` bit is clear, as ``length`` is when the ``L`` bit is, so that the one header keeps one data model, as settled on :issue:`2625`; its ``make()`` takes a ``cid`` and sets the ``C`` bit. - :mod:`pcapkit.protocols.data` and :mod:`pcapkit.protocols.schema` export a - ``tls12_cid`` record's classes as ``DTLS_CIDCiphertext`` and the extension's - data model as ``TLS_ConnectionId`` (:issue:`2515`, :pr:`2631`, :pr:`2641`). + A ``tls12_cid`` record's schema and data model are ``DTLSCIDCiphertext``, in + the DTLS package, and the extension's data model is ``ConnectionId``, in the + SSL package's (:issue:`2515`, :pr:`2631`, :pr:`2641`). Each whole DTLS record in a UDP datagram is likewise a layer of its own -- a DTLS 1.2 record then a DTLS 1.3 unified-header record read ``DTLS 1.2:DTLS 1.3`` -- a unified-header record with no length running to the end of the datagram, and one that fails to decode is - :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the datagram + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` with the rest of the datagram (:issue:`2535`, :pr:`2603`). The content, handshake and extension types are :mod:`pcapkit.const.tls`, crawled from IANA (:issue:`2495`, :issue:`2514`, :pr:`2503`, :pr:`2595`). -* **Added** -- :class:`~pcapkit.protocols.application.sslv2.SSLv2`, a dissector +* **Added** -- :class:`~pcapkit.protocols.application.sslv2.protocol.SSLv2`, a dissector for SSL version 2 (draft-hickman-netscape-ssl-00), whose record layer shares nothing with TLS's (:issue:`2457`, :pr:`2466`). It is bound to no port: on TCP 443 a segment's first record that TLS rejects is tried as SSLv2, and kept only as a complete CLIENT-HELLO or SERVER-HELLO whose length fields - agree, so anything else stays :class:`~pcapkit.protocols.misc.raw.Raw`; a + agree, so anything else stays :class:`~pcapkit.protocols.misc.raw.protocol.Raw`; a kept one drops the rejected TLS trial's warnings (:issue:`2525`, :pr:`2537`). Each whole record after the first in a TCP segment, whatever its message type, is a further SSLv2 layer, the previous record's payload, as with TLS -- a ``CLIENT-HELLO`` then a ``CLIENT-MASTER-KEY`` read ``SSLv2:SSLv2`` -- and one that fails to decode is - :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` with the rest of the segment (:issue:`2567`, :pr:`2585`). Its constants are :mod:`pcapkit.const.sslv2`, from a hand-maintained table. * **Added** -- dissection of every whole HTTP/2 frame in a TCP segment: each - after the first, which was kept as :class:`~pcapkit.protocols.misc.raw.Raw`, - is a further :class:`~pcapkit.protocols.application.httpv2.HTTPv2` layer, the + after the first, which was kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`, + is a further :class:`~pcapkit.protocols.application.httpv2.protocol.HTTPv2` layer, the previous frame's payload, as with TLS -- a ``SETTINGS`` then a ``WINDOW_UPDATE`` frame read ``TCP:HTTP/2:HTTP/2`` (:issue:`2536`, :pr:`2609`). One that fails to decode is - :class:`~pcapkit.protocols.misc.raw.Raw` with the rest of the segment and + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` with the rest of the segment and the parser's ``error``, and a frame the segment ends inside stays - :class:`~pcapkit.protocols.misc.raw.Raw` with none. A parse told to stop at + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` with none. A parse told to stop at HTTP/2 keeps the rest of the segment raw, and :data:`~pcapkit.utilities.config.FRAME_LAYER_LIMIT` bounds how many frames are dissected. @@ -350,7 +349,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. that decode as ``packet``, run on its first read as an IP or TCP datagram's is, and :data:`None` for an incomplete message (:issue:`2531`, :pr:`2605`). Both run the classmethod - :meth:`DTLS.analyze_handshake ` + :meth:`DTLS.analyze_handshake ` -- ``packet`` through the reassembly class's own ``__protocol_type__`` -- which keeps a body that does not decode as raw octets, logging nothing (:issue:`2622`, :pr:`2634`). @@ -362,7 +361,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. pair, and return :data:`None` for the tuple of received runs ``strict=True`` reports for an incomplete one (:issue:`2581`, :pr:`2619`). -* **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo`, +* **Added** -- :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.protocol.IP_Pseudo`, the IPv4 (:rfc:`793`) or IPv6 (:rfc:`8200#section-8.1`) pseudo-header an upper-layer checksum covers, built from an IPv4 or IPv6 layer, explicit values, or both (:issue:`2471`, :pr:`2479`). From a layer it takes the final @@ -388,7 +387,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. alone. Construction computes a checksum the caller leaves out, unless ``PCAPKIT_CHECKSUM_COMPUTE`` is off. An IPv4 or IPv6 layer verifies and computes its upper layer's checksum over the - :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.IP_Pseudo` it builds, and + :class:`~pcapkit.protocols.misc.checksum.ip_pseudo.protocol.IP_Pseudo` it builds, and ``make()`` of TCP, UDP and ICMPv6 also takes a ``pseudo``. What checksum offload leaves on the sending host is ``NOT_COMPUTED`` rather than a mismatch; a fragment's TCP, UDP, ICMP, ICMPv6, IGMP or GRE checksum, which @@ -421,8 +420,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. IPX's packet-type map, Loopback's address-family map and SCTP's chunk, chunk-parameter and error-cause parser maps with the validation the other wrappers carry, and the - :meth:`IPX.register ` and - :meth:`Loopback.register ` + :meth:`IPX.register ` and + :meth:`Loopback.register ` classmethods the IPX and Loopback wrappers call (:issue:`2511`, :pr:`2521`). @@ -453,7 +452,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. reduces a multi-valued mapping to a call of its class on its item pairs; and :func:`~pcapkit.utilities.pickle.reduce_to`, which makes one instance reduce to a given call. :class:`~pcapkit.corekit.infoclass.Info`, - :class:`~pcapkit.protocols.schema.schema.Schema` and the reassembly engines + :class:`~pcapkit.protocols.schema.Schema` and the reassembly engines take ``ReduceAsProtocol2``, so on GraalPy they pickle under protocols 0 and 1, where 1.5.x raised ``TypeError: typing.Generic() takes no arguments``. ``pcapkit.corekit.fields.numbers._reduce_unregistered_member`` is removed @@ -461,29 +460,29 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- **a breaking change to** the IPv6 extension-header walk: a known extension header whose dedicated parser raises is now the upper layer, - parsed once as :class:`~pcapkit.protocols.misc.raw.Raw`, and the walk stops + parsed once as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`, and the walk stops there, as :rfc:`8200#section-4` has a node discard such a packet (:issue:`2477`, :pr:`2480`). It used to fall back to the generic - :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and walk on; + :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` and walk on; ``IPv6_Ext`` now parses only Shim6, which has no dedicated parser, and ``IPv6.__generic_ext_codes__``, the headers that fell back, is removed. The library no longer passes - :meth:`IPv6_Ext.read() ` + :meth:`IPv6_Ext.read() ` an ``error=``, so - :attr:`IPv6_Ext.error ` + :attr:`IPv6_Ext.error ` is :data:`None` unless a caller supplies one; the ``Raw`` layer's ``info.error`` holds the failure instead. * **Changed** -- **a breaking change to** parsing a CALIPSO, SMF_DPD or MPL - option in :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` or - :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts`: an Opt Data Len too + option in :class:`~pcapkit.protocols.internet.hopopt.protocol.HOPOPT` or + :class:`~pcapkit.protocols.internet.ipv6_opts.protocol.IPv6_Opts`: an Opt Data Len too short for the option raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x raised :exc:`~pcapkit.utilities.exceptions.FieldValueError`, so code catching ``FieldValueError`` for it catches ``ProtocolError`` instead, both being :exc:`ValueError`; inside IPv6 the header is kept as ``Raw`` with nothing logged, where 1.5.x logged it at ``CRITICAL`` (:issue:`2727`, :pr:`2737`). So do the length - callbacks of :mod:`pcapkit.protocols.schema.internet.ipv6_option` that size + callbacks of :mod:`pcapkit.protocols.internet.ipv6_ext.schema` that size those options -- ``calipso_pad_len``, ``mpl_opt_pad_len``, ``smf_i_dpd_id_len`` and ``smf_dpd_data_selector``. A capture cut after such an option's type octet reads as a short read, a CALIPSO or SMF_DPD option @@ -496,19 +495,19 @@ HTTP/1 and HTTP/2 classes under the RFC names. :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x read the fields it lacked as zeros, or dropped the octets past them, and rebuilt it longer than captured (:issue:`2669`, :pr:`2683`). - :class:`~pcapkit.protocols.internet.mh.MH` refuses a whole message whose + :class:`~pcapkit.protocols.internet.mh.protocol.MH` refuses a whole message whose Header Len is short of its type's fixed fields, a message :rfc:`6275#section-9.2` has discarded, so a Home Test with Header Len 1 raises ``MH: [Type 3] invalid header length: 1``, where its 24 octets rebuilt as 32. Inside IPv6 it is the upper layer, kept as ``Raw`` by the walk above, and the packet rebuilds as captured, where its 64 octets - rebuilt as 72. :class:`~pcapkit.protocols.transport.tcp.TCP` refuses a DSS + rebuilt as 72. :class:`~pcapkit.protocols.transport.tcp.protocol.TCP` refuses a DSS Length other than the one :rfc:`8684` gives its ``A``, ``a``, ``M`` and ``m`` flags, or that plus the 2-octet checksum with ``M`` set, so a DSS of Length 6 carrying a 4-octet Data ACK, which takes 8, raises ``TCP: [OptNo 30] invalid format``, where its 40-octet segment rebuilt as 44; inside IPv4 the segment is kept as - :class:`~pcapkit.protocols.misc.raw.Raw` and the packet rebuilds as + :class:`~pcapkit.protocols.misc.raw.protocol.Raw` and the packet rebuilds as captured. A capture cut inside an MH message whose Header Len is short of it rebuilds with the Header Len it declared, where 1.5.x wrote the one its type takes. A mobility option whose fields do not take its Length is @@ -526,12 +525,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. IPv6 the header is ``Raw`` and the packet rebuilds as captured. * **Changed** -- **a breaking change to** MH option data models. - :class:`~pcapkit.protocols.data.internet.mh.QualityOfServiceOption`, the + :class:`~pcapkit.protocols.internet.mh.data.QualityOfServiceOption`, the Quality-of-Service option's, gains a required ``tc_reserved`` after ``dscp``: the two reserved bits of the traffic class, which 1.5.x dropped, so a traffic class of ``0xB9`` rebuilt as ``0xB8`` (:issue:`2684`, :pr:`2700`). Building the model without it raises - :exc:`TypeError`, and :class:`~pcapkit.protocols.internet.mh.MH`'s + :exc:`TypeError`, and :class:`~pcapkit.protocols.internet.mh.protocol.MH`'s ``make()`` takes it, ``0`` by default. The MN-ID and Service Selection makers count the octets an identifier packs as, so an NAI of ``'usér@x'`` takes Length 8, where 1.5.x wrote 7 from its characters, and a service @@ -540,7 +539,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. CGA public key of more than 127 octets, as a 1024-bit RSA key is, parses through the long form of its DER length, where 1.5.x misread the length and raised :exc:`~pcapkit.utilities.exceptions.ProtocolError`. - :class:`~pcapkit.protocols.data.internet.mh.MobileNetworkPrefixOption`, the + :class:`~pcapkit.protocols.internet.mh.data.MobileNetworkPrefixOption`, the Mobile Network Prefix option's, gains a required ``prefix_length``, and its ``prefix`` is the :class:`~ipaddress.IPv6Address` captured, where it was an :class:`~ipaddress.IPv6Network`, so a prefix with bits set past its length, @@ -550,7 +549,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. 255, where 1.5.x ignored it and wrote the length ``prefix`` gave, 128 for an address; and it takes ``prefix`` as an address, a network or an :class:`int`, keeping any bits past the length. In - :class:`~pcapkit.protocols.data.internet.mh.TimestampOption`, ``timestamp`` + :class:`~pcapkit.protocols.internet.mh.data.TimestampOption`, ``timestamp`` is :data:`None` past the year 9999, ``pmip_timestamp`` holding the value, where 1.5.x raised :exc:`ValueError` for a time past it, or late enough in its last second to round up to the year 10000. It is read without summing @@ -559,18 +558,18 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Changed** -- **a breaking change to** construction: a checksum left out of ``make()`` is computed rather than zero, so the octets built change - (:issue:`2471`, :pr:`2543`). :class:`~pcapkit.protocols.internet.ipv4.IPv4` + (:issue:`2471`, :pr:`2543`). :class:`~pcapkit.protocols.internet.ipv4.protocol.IPv4` computes its header checksum, an IPv4 or IPv6 layer that of a TCP or UDP payload built without one, and - :class:`~pcapkit.protocols.application.ospf.OSPF` that of an OSPFv2 packet. + :class:`~pcapkit.protocols.application.ospf.protocol.OSPF` that of an OSPFv2 packet. A checksum given is written as given, and turning ``PCAPKIT_CHECKSUM_COMPUTE`` off restores zero. * **Changed** -- **a breaking change to** building a Quick-Start option in - :class:`~pcapkit.protocols.internet.ipv4.IPv4`, - :class:`~pcapkit.protocols.internet.hopopt.HOPOPT`, - :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` and - :class:`~pcapkit.protocols.transport.tcp.TCP`: a value its field cannot + :class:`~pcapkit.protocols.internet.ipv4.protocol.IPv4`, + :class:`~pcapkit.protocols.internet.hopopt.protocol.HOPOPT`, + :class:`~pcapkit.protocols.internet.ipv6_opts.protocol.IPv6_Opts` and + :class:`~pcapkit.protocols.transport.tcp.protocol.TCP`: a value its field cannot carry raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x built it (:issue:`2617`, :pr:`2630`). That is a negative or non-integer rate, a TCP rate from 1 to 79 kbps, a Request TTL outside 0 to 255, an @@ -584,9 +583,9 @@ HTTP/1 and HTTP/2 classes under the RFC names. as IPv4's does. * **Changed** -- **a breaking change to** building - :class:`~pcapkit.protocols.transport.tcp.TCP`, - :class:`~pcapkit.protocols.transport.udp.UDP` and - :class:`~pcapkit.protocols.transport.sctp.SCTP`: a header or option field + :class:`~pcapkit.protocols.transport.tcp.protocol.TCP`, + :class:`~pcapkit.protocols.transport.udp.protocol.UDP` and + :class:`~pcapkit.protocols.transport.sctp.protocol.SCTP`: a header or option field given an integer it cannot hold raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` naming the field, where 1.5.x wrote it masked to the field's width, and a fixed-size octet field @@ -602,12 +601,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. defines modulo 256, wraps as in 1.5.x. * **Changed** -- **a breaking change to** building - :class:`~pcapkit.protocols.internet.ipv4.IPv4`, - :class:`~pcapkit.protocols.internet.hopopt.HOPOPT`, - :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts`, - :class:`~pcapkit.protocols.internet.ipv6_route.IPv6_Route`, - :class:`~pcapkit.protocols.internet.mh.MH` and - :class:`~pcapkit.protocols.internet.hip.HIP`, as TCP's, UDP's and SCTP's + :class:`~pcapkit.protocols.internet.ipv4.protocol.IPv4`, + :class:`~pcapkit.protocols.internet.hopopt.protocol.HOPOPT`, + :class:`~pcapkit.protocols.internet.ipv6_opts.protocol.IPv6_Opts`, + :class:`~pcapkit.protocols.internet.ipv6_route.protocol.IPv6_Route`, + :class:`~pcapkit.protocols.internet.mh.protocol.MH` and + :class:`~pcapkit.protocols.internet.hip.protocol.HIP`, as TCP's, UDP's and SCTP's above: a header, option or parameter field given an integer it cannot hold, or ``True`` where it is wider than one bit, raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` naming the field, where @@ -655,7 +654,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``IPv4: invalid timestamp count: 10``, where 1.5.x cut it to fit. * **Changed** -- **a breaking change to** building - :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` blocks: a field given a value + :class:`~pcapkit.protocols.misc.pcapng.protocol.PCAPNG` blocks: a field given a value it cannot hold raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, where 1.5.x wrote it masked (:issue:`2614`, :pr:`2671`). The Section Header Block's Section Length takes ``-2**63`` to ``2**64 - 1``, ``-1`` and @@ -702,28 +701,58 @@ HTTP/1 and HTTP/2 classes under the RFC names. where 1.5.x wrote the code with an empty phrase. * **Changed** -- **a breaking change to** - :class:`~pcapkit.protocols.application.drarp.DRARP` and - :class:`~pcapkit.protocols.link.inarp.InARP`, which move from + :class:`~pcapkit.protocols.application.drarp.protocol.DRARP` and + :class:`~pcapkit.protocols.link.inarp.protocol.InARP`, which move from :mod:`pcapkit.protocols.application.rarp` to :mod:`pcapkit.protocols.application.drarp` and from :mod:`pcapkit.protocols.link.arp` to :mod:`pcapkit.protocols.link.inarp`, - with no re-export at the old paths, as a module holds one protocol class + with no re-export at the old paths, as a package holds one protocol class (:issue:`2529`, :pr:`2540`, :pr:`2596`). ``pcapkit.DRARP``, ``pcapkit.InARP``, their :mod:`pcapkit.protocols` exports and a parsed RARP's ``DRARP`` or ARP's ``InARP`` alias are unchanged. * **Changed** -- **a breaking change to** - :class:`~pcapkit.protocols.misc.no_payload.NoPayload`, which moves from + :class:`~pcapkit.protocols.misc.no_payload.protocol.NoPayload`, which moves from ``pcapkit.protocols.misc.null`` to :mod:`pcapkit.protocols.misc.no_payload`, and its data model and header schema from ``pcapkit.protocols.data.misc.null`` - and ``pcapkit.protocols.schema.misc.null`` to - :mod:`pcapkit.protocols.data.misc.no_payload` and - :mod:`pcapkit.protocols.schema.misc.no_payload`, with no re-export at the old - paths, as a protocol class takes its module's name (:issue:`2529`, + and ``pcapkit.protocols.schema.misc.null`` to that package's + :mod:`~pcapkit.protocols.misc.no_payload.data` and + :mod:`~pcapkit.protocols.misc.no_payload.schema`, with no re-export at the + old paths, as a protocol class takes its package's name (:issue:`2529`, :pr:`2636`). The class keeps its name, and ``pcapkit.NoPayload`` and the package exports of that name are unchanged. A frame pickled by 1.5.x fails - to load, as the pickle names the old module, though its pickled ``info`` - loads. + to load, as any 1.5.x pickle of a data model does, below. + +* **Changed** -- **a breaking change to** the layout of + :mod:`pcapkit.protocols`: each protocol is a package holding its + dissector, header schema and data model as ``protocol``, ``schema`` and + ``data`` modules, as :mod:`pcapkit.protocols.internet.ipv4` holds + :mod:`~pcapkit.protocols.internet.ipv4.protocol`, + :mod:`~pcapkit.protocols.internet.ipv4.schema` and + :mod:`~pcapkit.protocols.internet.ipv4.data`, where 1.5.x kept the schemas + and data models in the separate ``pcapkit.protocols.schema`` and + ``pcapkit.protocols.data`` trees (:issue:`2534`, :pr:`2753`). Those trees + and their aggregate exports are removed with no alias, so + ``import pcapkit.protocols.data.internet.ipv4`` and + ``from pcapkit.protocols.data import PCAPNG_IF_NameOption`` raise + :exc:`ImportError`. A package's ``__init__.py`` exports its protocol class + and binds each schema and data class as ``Schema_`` and + ``Data_``, so + ``from pcapkit.protocols.internet.ipv4 import IPv4, Data_IPv4, Schema_IPv4`` + reads as before, and ``pcapkit.protocols.misc.pcapng.Data_IF_NameOption`` + replaces the aggregate ``PCAPNG_IF_NameOption``. The other names of a 1.5.x + protocol module are in its package's ``protocol`` module, and a + ``versionremoved`` note in the API reference gives each one that moved + elsewhere. :mod:`pcapkit.protocols.schema` and + :mod:`pcapkit.protocols.data` are modules holding the base classes, + :class:`~pcapkit.protocols.schema.Schema` and + :class:`~pcapkit.protocols.data.Data` among them, and the IPv6 option + helpers of ``pcapkit.protocols.schema.internet.ipv6_option`` are in + :mod:`pcapkit.protocols.internet.ipv6_ext.schema`. ``pcapkit.IPv4`` and the + :mod:`pcapkit.protocols` exports are unchanged. A data model pickled by + 1.5.x fails to load, as the pickle names its 1.5.x module, raising + ``ModuleNotFoundError: No module named 'pcapkit.protocols.data.internet'`` + say. * **Changed** -- **a breaking change to** the schema helpers that code outside the schema layer calls, which move to :mod:`pcapkit.corekit.schema`, a @@ -755,23 +784,22 @@ HTTP/1 and HTTP/2 classes under the RFC names. as an :class:`~pcapkit.corekit.multidict.OrderedMultiDict` without the ``packet``, ``frame_info`` and ``time_epoch`` attributes, which stay attributes, as in 1.5.x. A parsed TLS key log's - :attr:`TLSKeyLog.entries ` - is a :class:`~pcapkit.protocols.data.misc.pcapng.TLSKeyLogEntries` keyed by - label, which :mod:`pcapkit.protocols.data` exports as - ``PCAPNG_TLSKeyLogEntries``, where it was a :class:`dict`; - a mapping given for it is stored as one, and ``to_dict()`` exports it as an - ``OrderedMultiDict``, where it gave a :class:`dict`. Pickles of either load - across 1.5.x and 1.6.0 in both directions. ``AppType.__registries__`` is an + :attr:`TLSKeyLog.entries ` + is a :class:`~pcapkit.protocols.misc.pcapng.data.TLSKeyLogEntries` keyed by + label, which :mod:`pcapkit.protocols.misc.pcapng` binds as + ``Data_TLSKeyLogEntries``, where it was a :class:`dict`; a mapping given + for it is stored as one, and ``to_dict()`` exports it as an + ``OrderedMultiDict``, where it gave a :class:`dict`. ``AppType.__registries__`` is an ``Info`` keyed by :class:`~pcapkit.const.reg.apptype.apptype.TransportProtocol`, where 1.5.x held a :class:`dict` subclass, so setting a key raises :exc:`TypeError` (:issue:`2729`, :pr:`2735`). * **Changed** -- **a breaking change to** HTTP: parsing octets through - :class:`~pcapkit.protocols.application.http.HTTP`, building it from fields + :class:`~pcapkit.protocols.application.http.protocol.HTTP`, building it from fields and its ``from_data`` give the - :class:`~pcapkit.protocols.application.httpv1.HTTPv1` or - :class:`~pcapkit.protocols.application.httpv2.HTTPv2` member, never an + :class:`~pcapkit.protocols.application.httpv1.protocol.HTTPv1` or + :class:`~pcapkit.protocols.application.httpv2.protocol.HTTPv2` member, never an instance of the base, and so does a TCP or UDP payload on an HTTP port, whose data sits under ``httpv1`` or ``httpv2`` where it sat under ``http`` (:issue:`2524`, :pr:`2544`). Building it from fields takes ``version``, @@ -791,7 +819,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. version, and each version's own name, ``'HTTP/1'`` or ``'HTTPv1'`` say, selects it alone, where any HTTP layer matched ``'HTTPv1'`` and ``'HTTPv2'`` both. On the transport path a payload neither version recognises, a bare - connection preface included, is :class:`~pcapkit.protocols.misc.raw.Raw` + connection preface included, is :class:`~pcapkit.protocols.misc.raw.protocol.Raw` with no ``error``, where it carried ``'HTTP: unknown version'`` or ``'HTTP/2: connection preface with no frame'``; a direct ``HTTP(...)`` on it still raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`. One that @@ -850,7 +878,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. on first use, as TCP's and UDP's did in 1.5.x, where importing :mod:`pcapkit.const` loaded both. -* **Changed** -- :class:`~pcapkit.protocols.transport.sctp.SCTP`'s chunk makers +* **Changed** -- :class:`~pcapkit.protocols.transport.sctp.protocol.SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as ``SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''`` and @@ -864,10 +892,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. EtherType, TransType or AppType registry wherever one names it, and a class an AppType member names returns that member's port as an :class:`int` (:issue:`2627`, :pr:`2652`). So - :class:`~pcapkit.protocols.link.l2tpv2.L2TPv2`, - :class:`~pcapkit.protocols.application.ftp.FTP`, - :class:`~pcapkit.protocols.application.ftp_data.FTP_DATA`, the HTTP classes - and :class:`~pcapkit.protocols.application.ngap.NGAP`, which raised in 1.5.x, + :class:`~pcapkit.protocols.link.l2tpv2.protocol.L2TPv2`, + :class:`~pcapkit.protocols.application.ftp.protocol.FTP`, + :class:`~pcapkit.protocols.application.ftp_data.protocol.FTP_DATA`, the HTTP classes + and :class:`~pcapkit.protocols.application.ngap.protocol.NGAP`, which raised in 1.5.x, return 1701, 21, 20, 80 and 38412, and TLS, SSLv3 and DTLS the ports TCP and UDP dispatch them at, 443 and 5684; Radiotap, FRAD and GRE return their LinkType or TransType member. SSLv2 and the SSL base, which nothing @@ -879,7 +907,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. :pr:`2539`, :pr:`2541`, :pr:`2613`). * **Changed** -- the house conventions record that a protocol class takes its - module's name, one to a module (:issue:`2529`, :pr:`2542`); that a + package's name, one to a package (:issue:`2529`, :pr:`2542`); that a versioned family's base dispatches on every path, so parsing, building and ``from_data`` give a member rather than the base (:issue:`2524`, :pr:`2544`); that a docstring or prose block carries one ``versionadded`` @@ -927,8 +955,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. names one (:issue:`2574`, :pr:`2639`); HTTP's base follows that rule, its ``from_data`` reading the data model's class, since the HTTP data models carry no ``version`` (:issue:`2628`, :pr:`2651`). The member-order - convention's reference class is :class:`~pcapkit.protocols.link.arp.ARP`, - where it was :class:`~pcapkit.protocols.link.ethernet.Ethernet`, whose MAC + convention's reference class is :class:`~pcapkit.protocols.link.arp.protocol.ARP`, + where it was :class:`~pcapkit.protocols.link.ethernet.protocol.Ethernet`, whose MAC address helpers moved to the base class (:issue:`2614`, :pr:`2638`). A protocol's ``__index__`` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its @@ -954,7 +982,11 @@ HTTP/1 and HTTP/2 classes under the RFC names. :func:`~pcapkit.utilities.importlib.lazy_exports`, and a crawler never writes a package's ``__init__.py`` (:issue:`2729`, :pr:`2735`). A next-layer refusal in a payload a layer rebuilds is quiet, as the layer's own parse - reports it once (:issue:`2711`, :pr:`2738`). + reports it once (:issue:`2711`, :pr:`2738`). A protocol is a package of + ``protocol``, ``schema`` and ``data`` modules, whose ``__init__.py`` exports + its protocol class and binds its schema and data classes as ``Schema_*`` + and ``Data_*`` outside ``__all__``, which a ``[tool.isort]`` section keeps + isort off (:issue:`2534`, :pr:`2753`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/`` @@ -985,7 +1017,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. JUnit seconds to run in under 5 minutes (:issue:`2713`, :pr:`2720`). Its tests' timeout is 180 minutes, where it was 110, and the internet and application protocol shards are split too, as a slower runner ran out of - time with no worker lost (:issue:`2739`, :pr:`2741`). + time with no worker lost (:issue:`2739`, :pr:`2741`). The ``latest`` leg's + ``LATEST`` test tier runs in a job of its own, in halves, and a + ``Coverage`` job combines the coverage legs' parts, posting no total unless + every part arrived (:issue:`2751`, :pr:`2754`). The plain unittest ordering + leg runs the rest of ``tests/protocols`` in two halves, as the cell had + reached its 20-minute step cap with no test failed (:issue:`2751`). * **Changed** -- once a release's milestone is closed while ``__version__`` still names that release, the milestone workflow gives an item the lowest @@ -1021,7 +1058,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- under ``beholder``, a next layer captured at full length whose parse raised, such as a malformed TCP header, reported its warnings before - being replaced with :class:`~pcapkit.protocols.misc.raw.Raw`. Its trial now + being replaced with :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. Its trial now holds them however much was captured and drops them with the layer, so a kept layer's warnings follow those of the layers it carries (:issue:`2516`, :pr:`2519`). A :exc:`~pcapkit.utilities.exceptions.ProtocolError` built @@ -1047,7 +1084,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- selecting ``'IPv6_Ext'`` matched none of the dedicated extension headers: - :meth:`IPv6_Ext.id() ` now + :meth:`IPv6_Ext.id() ` now lists its direct concrete subclasses -- HOPOPT, IPv6_Route, IPv6_Frag, ESP, AH, IPv6_Opts, MH and HIP -- each of which keeps its own name alone, as AH and ESP do under IPsec (:issue:`2462`, :pr:`2464`). @@ -1055,7 +1092,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- a membership test such as ``'ESP' in layer`` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised :exc:`AttributeError`. - The layer now holds :class:`~pcapkit.protocols.misc.no_payload.NoPayload`, the + The layer now holds :class:`~pcapkit.protocols.misc.no_payload.protocol.NoPayload`, the upper layer belonging to the enclosing IPv6 header (:issue:`2484`, :issue:`2487`, :pr:`2486`, :pr:`2488`). @@ -1085,14 +1122,14 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- an HTTP/2 ``SETTINGS`` frame the capture cut short inside its settings raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` from the list of settings, so a segment carrying one was kept as - :class:`~pcapkit.protocols.misc.raw.Raw`. It parses, with the octets its + :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. It parses, with the octets its settings lack read as zeros, as a cut ``DATA`` frame's data is, and rebuilds as captured (:issue:`2659`, :issue:`2686`, :pr:`2666`, :pr:`2701`). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where ``from_data`` gave the preface's first octets, as many as were captured of the frame (:issue:`2665`, :pr:`2675`). -* **Fixed** -- an :class:`~pcapkit.protocols.transport.sctp.SCTP` packet the +* **Fixed** -- an :class:`~pcapkit.protocols.transport.sctp.protocol.SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's ``length``, rebuilt through ``from_data`` without the octets after its last whole chunk, which its @@ -1107,10 +1144,10 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- a capture cut inside or in front of an option area reads as a short read, as a cut elsewhere in a header does (:issue:`2694`, - :pr:`2731`). So an :class:`~pcapkit.protocols.internet.mh.MH` message cut + :pr:`2731`). So an :class:`~pcapkit.protocols.internet.mh.protocol.MH` message cut inside its options or reserved octets, and a - :class:`~pcapkit.protocols.internet.hopopt.HOPOPT` or - :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts` header cut short, + :class:`~pcapkit.protocols.internet.hopopt.protocol.HOPOPT` or + :class:`~pcapkit.protocols.internet.ipv6_opts.protocol.IPv6_Opts` header cut short, parse alone or inside IPv6 and rebuild as captured, logging nothing, where 1.5.x raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` or ``ProtocolError`` logged at ``CRITICAL``, as @@ -1123,7 +1160,7 @@ HTTP/1 and HTTP/2 classes under the RFC names. odd length, raises ``ProtocolError``, quiet inside IPv6, where 1.5.x raised ``FieldValueError: Field fid has invalid length.`` logged at ``CRITICAL``. -* **Fixed** -- :class:`~pcapkit.protocols.transport.tcp.TCP`'s maker, given an +* **Fixed** -- :class:`~pcapkit.protocols.transport.tcp.protocol.TCP`'s maker, given an option as raw octets, raised :exc:`~pcapkit.utilities.exceptions.EnumValueError` for a kind its registry does not name, such as 255, and :exc:`IndexError` for empty octets. The kind resolves as a ``(code, arguments)`` pair's does, so @@ -1147,20 +1184,20 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``ENCRYPTED`` parameter takes its cipher from the ``HIP_CIPHER`` before it, where 1.5.x warned ``HIP: [ParamNo 641] missing HIP_CIPHER parameter`` and read an AES IV as encrypted data. MH's CGA Parameters maker builds a raw - extension in a :class:`~pcapkit.protocols.data.internet.mh.CGAParameter` + extension in a :class:`~pcapkit.protocols.internet.mh.data.CGAParameter` data model as its ``(type, arguments)`` pair builds, where 1.5.x raised ``AttributeError: 'bytes' object has no attribute 'type'``, and a - :class:`~pcapkit.protocols.schema.internet.mh.CGAParameter` schema without + :class:`~pcapkit.protocols.internet.mh.schema.CGAParameter` schema without ``public_key_test``, leaving the schema given unchanged, where 1.5.x raised ``KeyError: 'public_key_test'`` (:issue:`2725`, :pr:`2732`). -* **Fixed** -- a :class:`~pcapkit.protocols.misc.pcapng.PCAPNG` Decryption +* **Fixed** -- a :class:`~pcapkit.protocols.misc.pcapng.protocol.PCAPNG` Decryption Secrets Block that ``make()`` built from a parsed ZigBee NWK or APS key shorter than its fixed width was corrupt: it declared that width, 18 or 22 octets, as its Secrets Length and sized the block from it, but wrote only the octets held, so a 5-octet NWK key built a 27-octet block declaring 40, where 28 were captured. It is written as captured, and - ``pcapkit.protocols.schema.misc.pcapng.zigbee_key_held`` returns a schema + ``pcapkit.protocols.misc.pcapng.schema.zigbee_key_held`` returns a schema whose :class:`bytes` and :func:`len` are those of the octets held, where they were the fixed width's (:issue:`2688`, :pr:`2697`). Rebuilt by ``from_data`` from the block's :class:`dict`, a TLS or WireGuard key log was written from its From fb56eae2db4c004dc7ed7f40c3db62059ea1fa62 Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Wed, 4 Nov 2026 09:08:16 -0500 Subject: [PATCH 26/27] docs(changelog): fold #2755, #2758 and #2759 (#2449) - Fixed: a cut header's rebuild keeps the next layer its parse had, where 1.5.x rebuilt a cut Ethernet header's Raw as NoPayload, and that Raw carries no error (#2759), a new entry; old values measured against v1.5.1. - Amended unreleased entries (#2758): a cut list of schema items ends with the data, the item the cut lands inside zero-filled and none past it, in the ListField, IGMP and HTTP/2 SETTINGS entries, the last also rebuilding at its declared Length or a recomputed one once edited. - Folded (#2755): a counted item the data ends inside reads as a quiet short read, into the ListField and IGMP entries. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 8 ++-- docs/source/changelog/1.6.0.rst | 67 ++++++++++++++++++++++----------- 2 files changed, 49 insertions(+), 26 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 05f4a92a17..68030f3eb0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,7 +16,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Added** -- `ICMPv4` ([RFC 792](https://datatracker.ietf.org/doc/html/rfc792)) and `ICMPv6` ([RFC 4443](https://datatracker.ietf.org/doc/html/rfc4443)), on an abstract `ICMP` base, for IP protocols 1 and 58, which were kept as `Raw` ([#2436](https://github.com/JarryShaw/PyPCAPKit/issues/2436), [#2446](https://github.com/JarryShaw/PyPCAPKit/pull/2446)); a protochain names `ICMPv6` where it named `IPv6_ICMP`. Echo, Destination Unreachable, Time Exceeded and Parameter Problem bodies are decoded, and ICMPv6's Packet Too Big; every other type, Neighbor Discovery and MLD included, keeps its body raw. ICMPv4 takes the plain `ICMP` of [RFC 792](https://datatracker.ietf.org/doc/html/rfc792) as its `id()` and alias. `register_icmp_type()` registers a message-body parser. The type and code registries are `pcapkit.const.icmp` and `pcapkit.const.icmpv6`, crawled from IANA. -- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). The octets it lacks read as zeros, the group records included, and a Report's header schema cut inside the Additional Data past its records packs back as captured ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2687](https://github.com/JarryShaw/PyPCAPKit/issues/2687), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). A Report parsed on its own sizes its group records by its record count, so one cut before the records it counts reads `TRUNCATED`, as it does behind IPv4, holding the records captured and none made up, and a count no IP payload could carry raises `ProtocolError` ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. +- **Added** -- IGMPv1 ([RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112)), IGMPv2 ([RFC 2236](https://datatracker.ietf.org/doc/html/rfc2236)) and IGMPv3 ([RFC 3376](https://datatracker.ietf.org/doc/html/rfc3376)), as `IGMPv1`, `IGMPv2` and `IGMPv3`, each named after its module ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)), on an abstract `IGMP` base bound at IP protocol 2, which picks the version from the message type alone except for a Membership Query: one of 8 octets is IGMPv1 if its Max Resp Time is zero and IGMPv2 otherwise, one of 12 or more IGMPv3 ([#2439](https://github.com/JarryShaw/PyPCAPKit/issues/2439), [#2448](https://github.com/JarryShaw/PyPCAPKit/issues/2448), [#2444](https://github.com/JarryShaw/PyPCAPKit/pull/2444), [#2465](https://github.com/JarryShaw/PyPCAPKit/pull/2465)). An IGMPv3 Query carries its source list and a Membership Report its group records, and an IGMPv3 Query's Max Resp Code and QQIC are decoded from their floating-point form. An IGMPv3 message the capture cut short, a Report cut inside its group records included, parses and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's `make()` takes `num_group_records`, from the record count to `0xFFFF` ([#2650](https://github.com/JarryShaw/PyPCAPKit/issues/2650), [#2660](https://github.com/JarryShaw/PyPCAPKit/issues/2660), [#2662](https://github.com/JarryShaw/PyPCAPKit/pull/2662), [#2672](https://github.com/JarryShaw/PyPCAPKit/pull/2672)). The octets it lacks read as zeros, in the group record the cut lands inside included, and a Report's header schema cut inside the Additional Data past its records packs back as captured ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2687](https://github.com/JarryShaw/PyPCAPKit/issues/2687), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)); no group record past the cut is read, however the Report is framed ([#2756](https://github.com/JarryShaw/PyPCAPKit/issues/2756), [#2758](https://github.com/JarryShaw/PyPCAPKit/pull/2758)). A Report parsed on its own sizes its group records by its record count, so one cut before the records it counts reads `TRUNCATED`, as it does behind IPv4, holding the records captured and none made up, and a count no IP payload could carry raises `ProtocolError` ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)); one cut inside a counted record reads quietly ([#2749](https://github.com/JarryShaw/PyPCAPKit/issues/2749), [#2755](https://github.com/JarryShaw/PyPCAPKit/pull/2755)). Any other message, a 9- to 11-octet Query included, is kept as `Raw`. IGMPv1 takes the plain `IGMP` of [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112) as its `id()` and alias, and IGMPv2's and IGMPv3's `id()` are their own names alone, `('IGMPv2',)` and `('IGMPv3',)`; the base's is `('IGMP', 'IGMPv2', 'IGMPv3')`, each member's canonical name ([#2512](https://github.com/JarryShaw/PyPCAPKit/issues/2512), [#2506](https://github.com/JarryShaw/PyPCAPKit/pull/2506), [#2518](https://github.com/JarryShaw/PyPCAPKit/pull/2518)). IGMPv2's and IGMPv3's `name` carries the version, `'Internet Group Management Protocol version 2'` say, while IGMPv1's, as in [RFC 1112](https://datatracker.ietf.org/doc/html/rfc1112), does not ([#2558](https://github.com/JarryShaw/PyPCAPKit/issues/2558), [#2590](https://github.com/JarryShaw/PyPCAPKit/pull/2590)). The registries are `pcapkit.const.igmp`: the types crawled from IANA, the group record types from a hand-maintained table. - **Added** -- construction through a versioned family's base: `IP` and `L2TP`, which raised `TypeError` in 1.5.x, and the `ICMP`, `IGMP` and `LinuxSLL` bases above give a member, never an instance of the base, on parsing octets, building from fields and `from_data` ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)). The `version` keyword names the member. Without it, IP's, IGMP's and L2TP's octets give the member they describe, and `from_data` the one its data names, IGMP's data models carrying a `version` for it. Building from fields without it raises `ProtocolError`, whose message asks for `version`, and so do ICMP and Linux cooked-capture octets, which carry no version; their `from_data` on a `dict` raises it too, as does a version a family has no member for, such as L2TP's `3`. A member built directly takes `version` as well, which must be its own. HTTP's base follows the same rules, a breaking change given below ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). IPv4 and IPv6 check the Version field before the rest of the header, so octets of the other version raise `[IPv4] invalid version: 6` say, where IPv4's error named whichever field they broke first. @@ -120,9 +120,9 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- a membership test such as `'ESP' in layer` on an extension header parsed by the IPv6 header's walk -- AH, HOPOPT, IPv6_Route, IPv6_Frag, IPv6_Opts, MH, HIP or the generic IPv6_Ext -- raised `AttributeError`. The layer now holds `NoPayload`, the upper layer belonging to the enclosing IPv6 header ([#2484](https://github.com/JarryShaw/PyPCAPKit/issues/2484), [#2487](https://github.com/JarryShaw/PyPCAPKit/issues/2487), [#2486](https://github.com/JarryShaw/PyPCAPKit/pull/2486), [#2488](https://github.com/JarryShaw/PyPCAPKit/pull/2488)). -- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside zero-filled to its length, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). Each item filled is a schema object rather than a zero octet, so `ZERO_PAD_BUDGET_RATIO` bounds the fill from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. If the items then fail, the fill stays charged, as a field's does, and the octets it counted as read are given back ([#2705](https://github.com/JarryShaw/PyPCAPKit/issues/2705), [#2706](https://github.com/JarryShaw/PyPCAPKit/pull/2706)). A `ListField` given a `count` reports a list whose octets hold fewer items than it counts as a short read, the absent items never made up ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. +- **Fixed** -- `ListField.unpack` reads a list of schema items the data ends inside up to the end of the data, the item the cut lands inside read with its missing octets as zeros, as `FieldBase.unpack` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised `FieldValueError` for consuming nothing ([#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). No item that would start past the captured octets is read, so a cut list holds no zero item the capture never held ([#2756](https://github.com/JarryShaw/PyPCAPKit/issues/2756), [#2758](https://github.com/JarryShaw/PyPCAPKit/pull/2758)). The fill is of schema objects rather than zero octets, so `ZERO_PAD_BUDGET_RATIO` bounds it from its first octet, with neither the free band of `MAX_ZERO_PAD_SHORTFALL` nor a one-off allowance, the zeros left unread going back to the budget, and a cut list declaring more than `MAX_ZERO_PAD_LENGTH` is refused. If the items then fail, the fill stays charged, as a field's does, and the octets it counted as read are given back ([#2705](https://github.com/JarryShaw/PyPCAPKit/issues/2705), [#2706](https://github.com/JarryShaw/PyPCAPKit/pull/2706)). A `ListField` given a `count` reports a list whose octets hold fewer items than it counts as a short read, the absent items never made up ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)), and reads an item it counts that the data ends inside as a quiet short read ([#2749](https://github.com/JarryShaw/PyPCAPKit/issues/2749), [#2755](https://github.com/JarryShaw/PyPCAPKit/pull/2755)). A TCP `SACK` option running past the data raises `ProtocolError`, `Field options has an option that runs past the end of the data: ...`, where it raised that `FieldValueError`, `Field sack has an item that consumed no data: ...`. -- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses, with the octets its settings lack read as zeros, as a cut `DATA` frame's data is, and rebuilds as captured ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). +- **Fixed** -- an HTTP/2 `SETTINGS` frame the capture cut short inside its settings raised `FieldValueError` from the list of settings, so a segment carrying one was kept as `Raw`. It parses, the setting the cut lands inside read with its missing octets as zeros, as a cut `DATA` frame's data is, and no setting past it, and rebuilds as captured ([#2659](https://github.com/JarryShaw/PyPCAPKit/issues/2659), [#2686](https://github.com/JarryShaw/PyPCAPKit/issues/2686), [#2666](https://github.com/JarryShaw/PyPCAPKit/pull/2666), [#2701](https://github.com/JarryShaw/PyPCAPKit/pull/2701)), at its declared Length, or at one recomputed from its settings once they are edited ([#2756](https://github.com/JarryShaw/PyPCAPKit/issues/2756), [#2758](https://github.com/JarryShaw/PyPCAPKit/pull/2758)). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where `from_data` gave the preface's first octets, as many as were captured of the frame ([#2665](https://github.com/JarryShaw/PyPCAPKit/issues/2665), [#2675](https://github.com/JarryShaw/PyPCAPKit/pull/2675)). - **Fixed** -- an `SCTP` packet the capture cut short of the length declared for it, by an IPv4 Total Length, an IPv6 Payload Length or a caller's `length`, rebuilt through `from_data` without the octets after its last whole chunk, which its header schema's `bytes()` counted twice. It rebuilds as captured, as one parsed with no length declared did ([#2673](https://github.com/JarryShaw/PyPCAPKit/issues/2673), [#2679](https://github.com/JarryShaw/PyPCAPKit/pull/2679)). Its header schema cut inside the padding of its last chunk, parameter or cause, or such an item's own schema cut inside its padding, packs back as captured, where 1.5.x packed it at full width, 34 octets in and 36 out; and a header schema cut inside an option area packs back the octets read, where a TCP header schema of data offset 8 cut 25 octets in gave 32 octets from `bytes()` and 23 from `pack()` ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2685](https://github.com/JarryShaw/PyPCAPKit/issues/2685), [#2682](https://github.com/JarryShaw/PyPCAPKit/pull/2682)). @@ -134,6 +134,8 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Fixed** -- rebuilding a layer through `from_data` reported its payload's messages twice, once from the payload's own rebuild and again from the layer's parse of it: in 1.5.x, `Ethernet.from_data` of a frame cut inside its TCP header logged the refusal twice, where the parse logged it once. Each is reported once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). IPv4 records a cut inside the payload length it declares as its `__short_read__`, where 1.5.x recorded none, and so do Ethernet, GRE, ICMP, IGMP and a TSO IPv4 layer for a cut at the end of their frame. +- **Fixed** -- a header the capture cut short parsed with a `Raw` next layer but rebuilt through `from_data` with a `NoPayload` one, from the same octets: in 1.5.x an `Ethernet` header cut inside its type field parsed as `Ethernet:Raw` and rebuilt as `Ethernet:NoPayload`, as did ARP's family, AH, HIP, the IPv6 extension headers, MH, VLAN, L2TPv2 and Loopback. The rebuild keeps the next layer the parse had, GRE's, FRAD's and the Linux cooked-capture headers' included ([#2717](https://github.com/JarryShaw/PyPCAPKit/issues/2717), [#2759](https://github.com/JarryShaw/PyPCAPKit/pull/2759)). A next layer a cut header leaves no octets is `Raw` without a dispatch on the code the cut left as zeros, and carries no `error`, where 1.5.x's carried the failed dispatch's, as `Field padding resolved to a negative length; template='-20s'`. + - **Fixed** -- the `json`, `plist` and `tree` dumps of an `Info`, and of its `to_dict()` export, left out a field named like one of its builtin names, such as `items`, `keys` or a reassembled datagram's `packet`. Each is dumped under its own name, and a reassembled datagram's `packet` as its parsed layer's `info` ([#2733](https://github.com/JarryShaw/PyPCAPKit/issues/2733), [#2736](https://github.com/JarryShaw/PyPCAPKit/pull/2736)). - **Fixed** -- importing `pcapkit.vendor.ipx`, or running its packet or socket crawler, without `requests` installed lost an entry of `sys.path` on each failed attempt, so a `pkgutil.walk_packages()` retrying it drained `sys.path` until it raised `IndexError`. The entry is now restored however the import ends ([#2589](https://github.com/JarryShaw/PyPCAPKit/issues/2589), [#2601](https://github.com/JarryShaw/PyPCAPKit/pull/2601)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 6d21352fe7..16e7659508 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -184,15 +184,18 @@ HTTP/1 and HTTP/2 classes under the RFC names. and rebuilds as captured, whether parsed on its own or carried by IPv4, and IGMPv3's ``make()`` takes ``num_group_records``, from the record count to ``0xFFFF`` (:issue:`2650`, :issue:`2660`, :pr:`2662`, :pr:`2672`). The - octets it lacks read as zeros, the group records included, and a Report's - header schema cut inside the Additional Data past its records packs back as - captured (:issue:`2686`, :issue:`2687`, :pr:`2701`). A Report parsed on its - own sizes its group records by its record count, so one cut before the - records it counts reads ``TRUNCATED``, as it does behind IPv4, holding the - records captured and none made up, and a count no IP payload could carry - raises :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2711`, - :pr:`2738`). Any other message, a 9- to - 11-octet Query included, is kept as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. + octets it lacks read as zeros, in the group record the cut lands inside + included, and a Report's header schema cut inside the Additional Data past + its records packs back as captured (:issue:`2686`, :issue:`2687`, + :pr:`2701`); no group record past the cut is read, however the Report is + framed (:issue:`2756`, :pr:`2758`). A Report parsed on its own sizes its + group records by its record count, so one cut before the records it counts + reads ``TRUNCATED``, as it does behind IPv4, holding the records captured + and none made up, and a count no IP payload could carry raises + :exc:`~pcapkit.utilities.exceptions.ProtocolError` (:issue:`2711`, + :pr:`2738`); one cut inside a counted record reads quietly (:issue:`2749`, + :pr:`2755`). Any other message, a 9- to 11-octet Query included, is kept as + :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. IGMPv1 takes the plain ``IGMP`` of :rfc:`1112` as its ``id()`` and alias, and IGMPv2's and IGMPv3's ``id()`` are their own names alone, ``('IGMPv2',)`` and ``('IGMPv3',)``; the base's is ``('IGMP', 'IGMPv2', 'IGMPv3')``, each @@ -1097,22 +1100,26 @@ HTTP/1 and HTTP/2 classes under the RFC names. :issue:`2487`, :pr:`2486`, :pr:`2488`). * **Fixed** -- :meth:`ListField.unpack ` - reads a list of schema items the data ends inside zero-filled to its length, + reads a list of schema items the data ends inside up to the end of the + data, the item the cut lands inside read with its missing octets as zeros, as :meth:`FieldBase.unpack ` reads any other field, so a schema cut inside one packs back as captured, where the first item past the data raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` for consuming nothing - (:issue:`2686`, :pr:`2701`). Each item filled is a schema object rather than - a zero octet, so :data:`~pcapkit.utilities.config.ZERO_PAD_BUDGET_RATIO` - bounds the fill from its first octet, with neither the free band of + (:issue:`2686`, :pr:`2701`). No item that would start past the captured + octets is read, so a cut list holds no zero item the capture never held + (:issue:`2756`, :pr:`2758`). The fill is of schema objects rather than zero + octets, so :data:`~pcapkit.utilities.config.ZERO_PAD_BUDGET_RATIO` + bounds it from its first octet, with neither the free band of :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_SHORTFALL` nor a one-off - allowance, and a cut list declaring more than - :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_LENGTH` is refused. If the - items then fail, the fill stays charged, as a field's does, and the octets - it counted as read are given back (:issue:`2705`, :pr:`2706`). A - ``ListField`` given a ``count`` reports a list whose octets hold fewer items - than it counts as a short read, the absent items never made up - (:issue:`2711`, :pr:`2738`). A TCP + allowance, the zeros left unread going back to the budget, and a cut list + declaring more than :data:`~pcapkit.utilities.config.MAX_ZERO_PAD_LENGTH` is + refused. If the items then fail, the fill stays charged, as a field's does, + and the octets it counted as read are given back (:issue:`2705`, + :pr:`2706`). A ``ListField`` given a ``count`` reports a list whose octets + hold fewer items than it counts as a short read, the absent items never + made up (:issue:`2711`, :pr:`2738`), and reads an item it counts that the + data ends inside as a quiet short read (:issue:`2749`, :pr:`2755`). A TCP ``SACK`` option running past the data raises :exc:`~pcapkit.utilities.exceptions.ProtocolError`, ``Field options has an option that runs past the end of the data: ...``, @@ -1122,9 +1129,12 @@ HTTP/1 and HTTP/2 classes under the RFC names. * **Fixed** -- an HTTP/2 ``SETTINGS`` frame the capture cut short inside its settings raised :exc:`~pcapkit.utilities.exceptions.FieldValueError` from the list of settings, so a segment carrying one was kept as - :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. It parses, with the octets its - settings lack read as zeros, as a cut ``DATA`` frame's data is, and rebuilds - as captured (:issue:`2659`, :issue:`2686`, :pr:`2666`, :pr:`2701`). A frame + :class:`~pcapkit.protocols.misc.raw.protocol.Raw`. It parses, the setting the + cut lands inside read with its missing octets as zeros, as a cut ``DATA`` + frame's data is, and no setting past it, and rebuilds as captured + (:issue:`2659`, :issue:`2686`, :pr:`2666`, :pr:`2701`), at its declared + Length, or at one recomputed from its settings once they are edited + (:issue:`2756`, :pr:`2758`). A frame of any type the capture cut short after the connection preface rebuilds as captured too, where ``from_data`` gave the preface's first octets, as many as were captured of the frame (:issue:`2665`, :pr:`2675`). @@ -1217,6 +1227,17 @@ HTTP/1 and HTTP/2 classes under the RFC names. recorded none, and so do Ethernet, GRE, ICMP, IGMP and a TSO IPv4 layer for a cut at the end of their frame. +* **Fixed** -- a header the capture cut short parsed with a ``Raw`` next layer + but rebuilt through ``from_data`` with a ``NoPayload`` one, from the same + octets: in 1.5.x an ``Ethernet`` header cut inside its type field parsed as + ``Ethernet:Raw`` and rebuilt as ``Ethernet:NoPayload``, as did ARP's + family, AH, HIP, the IPv6 extension headers, MH, VLAN, L2TPv2 and Loopback. + The rebuild keeps the next layer the parse had, GRE's, FRAD's and the Linux + cooked-capture headers' included (:issue:`2717`, :pr:`2759`). A next layer + a cut header leaves no octets is ``Raw`` without a dispatch on the code the + cut left as zeros, and carries no ``error``, where 1.5.x's carried the failed + dispatch's, as ``Field padding resolved to a negative length; template='-20s'``. + * **Fixed** -- the ``json``, ``plist`` and ``tree`` dumps of an :class:`~pcapkit.corekit.infoclass.Info`, and of its ``to_dict()`` export, left out a field named like one of its builtin names, such as ``items``, From ed294b4e69cdb4ea8dde98cbd185dd44556520ff Mon Sep 17 00:00:00 2001 From: Jarry Shaw Date: Wed, 4 Nov 2026 09:17:29 -0500 Subject: [PATCH 27/27] docs(changelog): fold #2757 (#2449) - Folded into the lazy-import entry (#2757): pcapkit.protocols, its layer packages, misc.pcap, misc.checksum and pcapkit import their protocol classes on first use, pcapkit.protocols.registry builds __proto__ on first read, and dir(pcapkit) lists none of the incidental names 1.5.1 listed (os, sys, ROOT, warnings, tbtrim), measured on a v1.5.1 export; the ruling into the conventions entry. - No mention of foundation.registry.protocols.protocol_registry, an import alias 1.5.1 neither exported nor documented, nor of an import time. - CHANGELOG.md regenerated; util/changelog_md.py --check passes. --- CHANGELOG.md | 4 ++-- docs/source/changelog/1.6.0.rst | 13 +++++++++++-- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 68030f3eb0..a081bc51a4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -84,7 +84,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- `pcapkit.foundation.registry.protocols` is a package with a submodule per layer -- `link`, `internet`, `transport`, `application` and `misc`, with the shared registrars in `protocol` -- and exports every public function they define, so each `register_*` imports from it and from `pcapkit.foundation.registry` as before ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A registrar's `__module__` names its submodule, and its log records carry that submodule's logger name, a child of the package's, so configuring the package's logger still applies. -- **Changed** -- a `pcapkit.const` package imports each of its enumeration modules on first use rather than with itself, and `pcapkit.const` its packages, through `pcapkit.utilities.importlib.lazy_exports()`, which builds a package's PEP 562 `__getattr__` and `__dir__` from its exports ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). Each package's `__all__`, `dir()` and members are as in 1.5.x. The SCTP and DCCP application-type registries load on first use, as TCP's and UDP's did in 1.5.x, where importing `pcapkit.const` loaded both. +- **Changed** -- a `pcapkit.const` package imports each of its enumeration modules on first use rather than with itself, and `pcapkit.const` its packages, through `pcapkit.utilities.importlib.lazy_exports()`, which builds a package's PEP 562 `__getattr__` and `__dir__` from its exports ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). Each package's `__all__`, `dir()` and members are as in 1.5.x. The SCTP and DCCP application-type registries load on first use, as TCP's and UDP's did in 1.5.x, where importing `pcapkit.const` loaded both. So do `pcapkit.protocols`, its layer packages, `pcap` and `checksum`, each importing a protocol's package when its class is first reached, and `pcapkit` its protocol classes, while a protocol's own package imports its modules with itself ([#2730](https://github.com/JarryShaw/PyPCAPKit/issues/2730), [#2757](https://github.com/JarryShaw/PyPCAPKit/pull/2757)). `pcapkit.protocols.__proto__` is built by `pcapkit.protocols.registry` when first read, and `dir(pcapkit)` lists no incidental names, where 1.5.x listed `os`, `sys`, `ROOT`, `warnings` and `tbtrim`. - **Changed** -- `SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause maker reserved octets that are not two, as `SCTP: invalid [Chunk 132] flags: must be 1 octet, not b''` and `SCTP: invalid [Cause 1] reserved: must be 2 octets, not b''`, where they read `SCTP: [Chunk 132] invalid flags: must be one octet, not b''` and `SCTP: [Cause 1] invalid reserved: must be two octets, not b''` ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A one-character `str` given as flags, which passed the check and failed to pack, is refused the same way. @@ -92,7 +92,7 @@ Wave 2 of the roadmap, per the contributing docs' enhancement-proposal page: the - **Changed** -- protocol class bodies are reordered to the member-order convention, with no change in behaviour, and a project test guards the order ([#2490](https://github.com/JarryShaw/PyPCAPKit/issues/2490), [#2491](https://github.com/JarryShaw/PyPCAPKit/pull/2491), [#2522](https://github.com/JarryShaw/PyPCAPKit/pull/2522), [#2523](https://github.com/JarryShaw/PyPCAPKit/pull/2523), [#2538](https://github.com/JarryShaw/PyPCAPKit/pull/2538), [#2539](https://github.com/JarryShaw/PyPCAPKit/pull/2539), [#2541](https://github.com/JarryShaw/PyPCAPKit/pull/2541), [#2613](https://github.com/JarryShaw/PyPCAPKit/pull/2613)). -- **Changed** -- the house conventions record that a protocol class takes its package's name, one to a package ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). A next-layer refusal in a payload a layer rebuilds is quiet, as the layer's own parse reports it once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). A protocol is a package of `protocol`, `schema` and `data` modules, whose `__init__.py` exports its protocol class and binds its schema and data classes as `Schema_*` and `Data_*` outside `__all__`, which a `[tool.isort]` section keeps isort off ([#2534](https://github.com/JarryShaw/PyPCAPKit/issues/2534), [#2753](https://github.com/JarryShaw/PyPCAPKit/pull/2753)). +- **Changed** -- the house conventions record that a protocol class takes its package's name, one to a package ([#2529](https://github.com/JarryShaw/PyPCAPKit/issues/2529), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); that a versioned family's base dispatches on every path, so parsing, building and `from_data` give a member rather than the base ([#2524](https://github.com/JarryShaw/PyPCAPKit/issues/2524), [#2544](https://github.com/JarryShaw/PyPCAPKit/pull/2544)); that a docstring or prose block carries one `versionadded` per version, without a body that only names the thing added ([#2530](https://github.com/JarryShaw/PyPCAPKit/issues/2530), [#2542](https://github.com/JarryShaw/PyPCAPKit/pull/2542)); and that each class-level registry map has a wrapper in `pcapkit.foundation.registry` ([#2511](https://github.com/JarryShaw/PyPCAPKit/issues/2511), [#2521](https://github.com/JarryShaw/PyPCAPKit/pull/2521)). `const` is a label added alongside a commit type rather than a type, in the pull request template and `CONTRIBUTING.md` too, and a span the transport registries assign to a service keeps its bare name, the mint criterion's exception to code-suffixing ([#2549](https://github.com/JarryShaw/PyPCAPKit/issues/2549), [#2550](https://github.com/JarryShaw/PyPCAPKit/issues/2550), [#2570](https://github.com/JarryShaw/PyPCAPKit/pull/2570)). The registry conventions record the exceptions to their case-folding rules -- the crawlers that re-case the names they generate, the `Method` constructor's fold, and `Command.get` and `Method.get` answering a name miss with an unregistered member -- and the folds that live outside any registry ([#2551](https://github.com/JarryShaw/PyPCAPKit/issues/2551), [#2552](https://github.com/JarryShaw/PyPCAPKit/issues/2552), [#2553](https://github.com/JarryShaw/PyPCAPKit/issues/2553), [#2556](https://github.com/JarryShaw/PyPCAPKit/issues/2556), [#2565](https://github.com/JarryShaw/PyPCAPKit/issues/2565), [#2582](https://github.com/JarryShaw/PyPCAPKit/issues/2582), [#2583](https://github.com/JarryShaw/PyPCAPKit/pull/2583)). They also record the SSL family's grouping, and that a family base is bound, and dispatches, only where the registry code cannot tell its members apart; that a pseudo-header protocol belongs in `pcapkit.protocols.misc`; that `pcapkit.utilities.compat` holds compatibility implementations and re-exports only, behind version gates; and how a ruling given in a pull request's review is cited ([#2557](https://github.com/JarryShaw/PyPCAPKit/issues/2557), [#2563](https://github.com/JarryShaw/PyPCAPKit/issues/2563), [#2599](https://github.com/JarryShaw/PyPCAPKit/pull/2599)). The test conventions record that a test class whose tests each need a fresh import purges in `setUp`, with a comment saying why, rather than importing once per class ([#2562](https://github.com/JarryShaw/PyPCAPKit/issues/2562), [#2611](https://github.com/JarryShaw/PyPCAPKit/pull/2611)). Where a registry's primary source is paywalled, the const-vendor pairing accepts a pinned secondary revision as a crawler's `DATA` once its values are cross-checked against independent implementations ([#2575](https://github.com/JarryShaw/PyPCAPKit/issues/2575), [#2607](https://github.com/JarryShaw/PyPCAPKit/pull/2607)). A context variable more than one module uses lives in `pcapkit.utilities.contextvars`, and one that only its own module uses stays private there ([#2616](https://github.com/JarryShaw/PyPCAPKit/issues/2616), [#2623](https://github.com/JarryShaw/PyPCAPKit/pull/2623)). Each `:issue:` and `:pr:` role in the documentation names a number of its own kind, a ruling given in a pull request's review cited by the issue that pull request closes, and a project test checks each role's kind against a snapshot of the tracker ([#2561](https://github.com/JarryShaw/PyPCAPKit/issues/2561), [#2626](https://github.com/JarryShaw/PyPCAPKit/pull/2626)). A registry wrapper goes in the submodule of `pcapkit.foundation.registry.protocols` for the layer of the class owning its map ([#2578](https://github.com/JarryShaw/PyPCAPKit/issues/2578), [#2635](https://github.com/JarryShaw/PyPCAPKit/pull/2635)). A protocol class owns the parse of a reassembled outcome, which each `pcapkit.foundation.analysis` function wraps without parsing of its own, and a project test fails on a module under `pcapkit.protocols` importing `pcapkit.foundation` outside `TYPE_CHECKING`, `pcapkit.foundation.registry.protocols` aside ([#2622](https://github.com/JarryShaw/PyPCAPKit/issues/2622), [#2634](https://github.com/JarryShaw/PyPCAPKit/pull/2634)). A versioned family's base takes its member from the `version` keyword, or else from the octets or the data model, never from a field, has no default member, and refuses octets that carry no version unless `version` names one ([#2574](https://github.com/JarryShaw/PyPCAPKit/issues/2574), [#2639](https://github.com/JarryShaw/PyPCAPKit/pull/2639)); HTTP's base follows that rule, its `from_data` reading the data model's class, since the HTTP data models carry no `version` ([#2628](https://github.com/JarryShaw/PyPCAPKit/issues/2628), [#2651](https://github.com/JarryShaw/PyPCAPKit/pull/2651)). The member-order convention's reference class is `ARP`, where it was `Ethernet`, whose MAC address helpers moved to the base class ([#2614](https://github.com/JarryShaw/PyPCAPKit/issues/2614), [#2638](https://github.com/JarryShaw/PyPCAPKit/pull/2638)). A protocol's `__index__` returns its LinkType, EtherType, TransType or AppType value ahead of any other registry's code, an AppType value as its port ([#2627](https://github.com/JarryShaw/PyPCAPKit/issues/2627), [#2652](https://github.com/JarryShaw/PyPCAPKit/pull/2652)). A protocol's table from a code to the layout read under it is a class-level registry map, with its `register_` classmethod and wrapper, never a module-level dictionary, and each layout is a concrete schema with typed fields, never built inside the map ([#2576](https://github.com/JarryShaw/PyPCAPKit/issues/2576), [#2657](https://github.com/JarryShaw/PyPCAPKit/pull/2657)). A `make()` field holding an enumeration takes the member or an `int` code, never a name, checked against the field's width and resolved with the `get` of an enumeration named at the call, never one read from the caller's frame, and no keyword group sits beside it ([#2594](https://github.com/JarryShaw/PyPCAPKit/issues/2594), [#2692](https://github.com/JarryShaw/PyPCAPKit/pull/2692)). A helper that code outside the schema layer calls lives in `pcapkit.corekit.schema`, a module per group, as does a helper in one protocol's schema module once code outside that protocol uses it and it is generic enough to reuse, with no re-export at its old path ([#2674](https://github.com/JarryShaw/PyPCAPKit/issues/2674), [#2704](https://github.com/JarryShaw/PyPCAPKit/pull/2704)). A protocol reads and rebuilds a capture cut through the schema's built-in short read, never an `unpack` override of its own or a length callable clamped to the octets present ([#2680](https://github.com/JarryShaw/PyPCAPKit/issues/2680), [#2694](https://github.com/JarryShaw/PyPCAPKit/issues/2694), [#2731](https://github.com/JarryShaw/PyPCAPKit/pull/2731)); a field callback leaves a value the cut left as zero to that short read, and refuses one read with `ProtocolError` ([#2727](https://github.com/JarryShaw/PyPCAPKit/issues/2727), [#2737](https://github.com/JarryShaw/PyPCAPKit/pull/2737)). A `pcapkit.const` package imports its enumerations on first use, through `lazy_exports()`, and a crawler never writes a package's `__init__.py` ([#2729](https://github.com/JarryShaw/PyPCAPKit/issues/2729), [#2735](https://github.com/JarryShaw/PyPCAPKit/pull/2735)). A next-layer refusal in a payload a layer rebuilds is quiet, as the layer's own parse reports it once ([#2711](https://github.com/JarryShaw/PyPCAPKit/issues/2711), [#2738](https://github.com/JarryShaw/PyPCAPKit/pull/2738)). A protocol is a package of `protocol`, `schema` and `data` modules, whose `__init__.py` exports its protocol class and binds its schema and data classes as `Schema_*` and `Data_*` outside `__all__`, which a `[tool.isort]` section keeps isort off ([#2534](https://github.com/JarryShaw/PyPCAPKit/issues/2534), [#2753](https://github.com/JarryShaw/PyPCAPKit/pull/2753)), and the packages above it import it on first use through `lazy_exports` ([#2730](https://github.com/JarryShaw/PyPCAPKit/issues/2730), [#2757](https://github.com/JarryShaw/PyPCAPKit/pull/2757)). - **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under `NotImplemented/` excepted, and the modules this release adds that had none -- `pcapkit.const.linux`, the `pcapkit.vendor.tls` and `pcapkit.vendor.sslv2` crawlers and the checksum packages -- now have one ([#2559](https://github.com/JarryShaw/PyPCAPKit/issues/2559), [#2584](https://github.com/JarryShaw/PyPCAPKit/pull/2584)). diff --git a/docs/source/changelog/1.6.0.rst b/docs/source/changelog/1.6.0.rst index 16e7659508..45222599ba 100644 --- a/docs/source/changelog/1.6.0.rst +++ b/docs/source/changelog/1.6.0.rst @@ -879,7 +879,15 @@ HTTP/1 and HTTP/2 classes under the RFC names. (:issue:`2729`, :pr:`2735`). Each package's ``__all__``, ``dir()`` and members are as in 1.5.x. The SCTP and DCCP application-type registries load on first use, as TCP's and UDP's did in 1.5.x, where importing - :mod:`pcapkit.const` loaded both. + :mod:`pcapkit.const` loaded both. So do :mod:`pcapkit.protocols`, its layer + packages, :mod:`~pcapkit.protocols.misc.pcap` and + :mod:`~pcapkit.protocols.misc.checksum`, each importing a protocol's package + when its class is first reached, and :mod:`pcapkit` its protocol classes, + while a protocol's own package imports its modules with itself + (:issue:`2730`, :pr:`2757`). :data:`pcapkit.protocols.__proto__` is built by + :mod:`pcapkit.protocols.registry` when first read, and ``dir(pcapkit)`` + lists no incidental names, where 1.5.x listed ``os``, ``sys``, ``ROOT``, + ``warnings`` and ``tbtrim``. * **Changed** -- :class:`~pcapkit.protocols.transport.sctp.protocol.SCTP`'s chunk makers refuse flags that are not one octet, and its Invalid Stream Identifier cause @@ -989,7 +997,8 @@ HTTP/1 and HTTP/2 classes under the RFC names. ``protocol``, ``schema`` and ``data`` modules, whose ``__init__.py`` exports its protocol class and binds its schema and data classes as ``Schema_*`` and ``Data_*`` outside ``__all__``, which a ``[tool.isort]`` section keeps - isort off (:issue:`2534`, :pr:`2753`). + isort off (:issue:`2534`, :pr:`2753`), and the packages above it import it + on first use through ``lazy_exports`` (:issue:`2730`, :pr:`2757`). * **Changed** -- a project test fails on a shipped module with no documentation entry, the placeholder sources under ``NotImplemented/``