refactor(protocols)!: group each protocol's protocol, schema and data into one package (#2534) (#2753)

The output of codemod-v2.py on 4f62f52, with no hand edits.

- protocols/<layer>/<proto>.py and its schema/ and data/ modules become
  protocols/<layer>/<proto>/{protocol,schema,data}.py; each package
  __init__ exports its class alone, eagerly, and binds each public class
  its schema and data modules define as Schema_<name> and Data_<name>,
  outside __all__, in a block per role under a "# Header Schemas" or
  "# Data Models" heading, as ruled in review of #2753. A [tool.isort]
  section in pyproject.toml keeps isort off these files, filter_files
  included; protocol-family-grouping.rst shows the layout and the
  doc-claims test pins every __init__ to it.
- The TLS record data models SSL reads move from tls/data.py to a new
  ssl/data.py, as pieces the family shares; tls/data.py keeps TLS.
- The schema/ and data/ trees and their aggregations go, with no alias:
  schema/schema.py is protocols/schema.py, data/data.py and data/protocol.py
  merge into protocols/data.py, and ipv6_option merges into ipv6_ext.schema.
- ProtocolBase.__init_subclass__ finds a class's schema and data in its own
  package and imports Raw's on demand.
- Imports, ModuleDescriptor strings, docs targets, paths and test specs are
  rewritten; released changelogs get every role target, 1.6.0 stays with
  #2483.
- versionremoved records every name v1.5.1 shipped that no longer resolves
  at its old path, read from the tag by a new util/shipped_names.py into a
  checked-in fixture; a re-export points at its defining module, and the
  address patterns #2614 removed unrecorded get their note too.
- The 1744 test gains TestShippedNames, which fails on any such name left
  unrecorded; test_public_api leaves the ruled aliases out of its unlisted
  names; tests that walked or pinned the old trees follow the new layout,
  two that pinned only the aggregations are deleted, and a new test pins
  the layout and the aliases.

All modules import, protocol models are unchanged, every unittest leg and
project guard passes, and a non-editable docs build is warning-free.

Author: JarryShaw

CONTRIBUTING.md

@@ -119,7 +119,7 @@ Mechanical points that decide real cases:
 - **A private base class must stay documented when a documented subclass carries
   `:show-inheritance:`.** Sphinx renders that subclass's `Bases:` line as a link into the private
   class's page, so dropping the directive breaks a link a public page renders. This keeps
-  `pcapkit.protocols.schema.misc.pcapng._OPT_Option` and the other per-block option bases with
+  `pcapkit.protocols.misc.pcapng.schema._OPT_Option` and the other per-block option bases with
   documented subclasses, plus `_IPField`, `_IPInterfaceField` and `_TextField`.
 - **Dunders keep their directives.** A `__dunder__` is reached through public syntax, not by name
   (`len()` calls `__len__`, `obj[key]` calls `__getitem__`), so overriding one changes behaviour a

docs/source/changelog/1.5.0.rst

@@ -53,7 +53,7 @@
 import pcapkit
 from pcapkit.corekit.infoclass import Info
 from pcapkit.corekit.sentinels import NO_VALUE
-from pcapkit.protocols.schema.schema import Schema
+from pcapkit.protocols.schema import Schema
 
 if TYPE_CHECKING:
     from types import FunctionType, ModuleType
@@ -122,9 +122,9 @@
 # These are deliberately opt-in per site rather than an automatic ``#NNN`` rule,
 # because no digit-keyed pattern can tell a citation from a packet-diagram label:
 # ``#\d{3}`` misses every two- and four-digit number, while ``#\d+`` catches the
-# one-digit RFC diagram labels in ``pcapkit/protocols/internet/hip.py``,
-# ``pcapkit/protocols/transport/sctp.py`` and
-# ``pcapkit/protocols/schema/internet/hip.py`` -- ``DH GROUP ID #1``,
+# one-digit RFC diagram labels in ``pcapkit/protocols/internet/hip/protocol.py``,
+# ``pcapkit/protocols/transport/sctp/protocol.py`` and
+# ``pcapkit/protocols/internet/hip/schema.py`` -- ``DH GROUP ID #1``,
 # ``Gap Ack Block #1`` and the like -- which are not references to anything. A role
 # nobody writes cannot corrupt them.
 #
@@ -446,7 +446,7 @@ def bind_guarded_names(mock_imports: 'List[str]', module: 'ModuleType') -> None:
     :exc:`NameError` for it and the schema class signatures fell back to the bare
     ``_VT``. The Python domain then found both
     :data:`pcapkit.corekit.multidict._VT` and
-    :data:`pcapkit.protocols.schema.schema._VT` for it, warned, and linked the
+    :data:`pcapkit.protocols.schema._VT` for it, warned, and linked the
     first, which is the wrong one. Binding what the library recorded restores
     what the releases before 3.14 did.
 

docs/source/conf.py

@@ -76,7 +76,7 @@ predated the crawler and had drifted from it in the ways cross-review of
   IANA counterpart and no reference anywhere in the codebase (IANA leaves 0
   inside its 0-19 ``Unassigned`` band, unnamed); it is dropped, and value 0
   now resolves to ``Unassigned`` like the rest of that band.
-  :class:`~pcapkit.protocols.application.ssl.SSL` never checked this member
+  :class:`~pcapkit.protocols.application.ssl.protocol.SSL` never checked this member
   by name -- its wire-type allowlist already excluded value 0 by construction
   -- so the sentinel's removal is behaviourally a no-op there too.
 * the committed ``HandshakeType`` cited RFC 5246, 6066 and 4680 for several

docs/source/contributing/conventions/const-vendor-pairing.rst

@@ -4,17 +4,17 @@ Extension-Header Base Classes
 -----------------------------
 
 Every IPv6 extension header in this package subclasses
-:class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext`. Some name a **second** base
+:class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext`. Some name a **second** base
 as well, and which ones do is a ruling rather than an accident. The owner ruled, in
-review of the rename that made :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` the
+review of the rename that made :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` the
 shared base (:issue:`917`), that a header usable *only* as an extension header inherits
-:class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and nothing else --
-:class:`~pcapkit.protocols.internet.ipv6_frag.IPv6_Frag` being the example -- while one
+:class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` and nothing else --
+:class:`~pcapkit.protocols.internet.ipv6_frag.protocol.IPv6_Frag` being the example -- while one
 that is usable as a standalone protocol in its own right inherits both
-:class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` and
+:class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` and
 :class:`~pcapkit.protocols.internet.internet.Internet` (or
-:class:`~pcapkit.protocols.internet.ipsec.IPsec`), as
-:class:`~pcapkit.protocols.internet.esp.ESP` does.
+:class:`~pcapkit.protocols.internet.ipsec.protocol.IPsec`), as
+:class:`~pcapkit.protocols.internet.esp.protocol.ESP` does.
 
 The family as it stands:
 
@@ -25,26 +25,26 @@ The family as it stands:
    * - Header
      - Bases
      - Classification
-   * - :class:`~pcapkit.protocols.internet.hopopt.HOPOPT`,
-       :class:`~pcapkit.protocols.internet.ipv6_route.IPv6_Route`,
-       :class:`~pcapkit.protocols.internet.ipv6_frag.IPv6_Frag`,
-       :class:`~pcapkit.protocols.internet.ipv6_opts.IPv6_Opts`,
-       :class:`~pcapkit.protocols.internet.mh.MH`
-     - :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext`
+   * - :class:`~pcapkit.protocols.internet.hopopt.protocol.HOPOPT`,
+       :class:`~pcapkit.protocols.internet.ipv6_route.protocol.IPv6_Route`,
+       :class:`~pcapkit.protocols.internet.ipv6_frag.protocol.IPv6_Frag`,
+       :class:`~pcapkit.protocols.internet.ipv6_opts.protocol.IPv6_Opts`,
+       :class:`~pcapkit.protocols.internet.mh.protocol.MH`
+     - :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext`
      - extension-header only
-   * - :class:`~pcapkit.protocols.internet.ah.AH`,
-       :class:`~pcapkit.protocols.internet.esp.ESP`
-     - :class:`~pcapkit.protocols.internet.ipsec.IPsec`,
-       :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext`
+   * - :class:`~pcapkit.protocols.internet.ah.protocol.AH`,
+       :class:`~pcapkit.protocols.internet.esp.protocol.ESP`
+     - :class:`~pcapkit.protocols.internet.ipsec.protocol.IPsec`,
+       :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext`
      - **also standalone**
-   * - :class:`~pcapkit.protocols.internet.hip.HIP`
-     - :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext`,
+   * - :class:`~pcapkit.protocols.internet.hip.protocol.HIP`
+     - :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext`,
        :class:`~pcapkit.protocols.internet.internet.Internet`
      - **also standalone**
 
-:class:`~pcapkit.protocols.internet.ipsec.IPsec` is itself an
+:class:`~pcapkit.protocols.internet.ipsec.protocol.IPsec` is itself an
 :class:`~pcapkit.protocols.internet.internet.Internet` subclass, which is the
-parenthetical :class:`~pcapkit.protocols.internet.ipsec.IPsec` alternative in the ruling:
+parenthetical :class:`~pcapkit.protocols.internet.ipsec.protocol.IPsec` alternative in the ruling:
 naming it satisfies the convention, and it is the right second base for a header whose
 standalone form is an IPsec one.
 
@@ -58,15 +58,15 @@ measurably so:
 
 .. code-block:: pycon
 
-   >>> from pcapkit.protocols.internet.ipv4 import IPv4
+   >>> from pcapkit.protocols.internet.ipv4.protocol import IPv4
    >>> from pcapkit.protocols.internet.internet import Internet
    >>> IPv4.__proto__ is Internet.__proto__
    True
 
 The protocol-number registry is **one shared object**, so *every* extension header
 is reachable as an IPv4 payload in this library --
-:class:`~pcapkit.protocols.internet.hopopt.HOPOPT` and
-:class:`~pcapkit.protocols.internet.ipv6_frag.IPv6_Frag` included. Registry
+:class:`~pcapkit.protocols.internet.hopopt.protocol.HOPOPT` and
+:class:`~pcapkit.protocols.internet.ipv6_frag.protocol.IPv6_Frag` included. Registry
 membership therefore says nothing at all about standalone-ness, and a classification
 derived from it would make every one of them standalone.
 
@@ -81,22 +81,22 @@ The IPv4-Payload Test
 So the census is read out of the specifications, on the owner's instruction: work
 through the RFCs to establish, for each defined IPv6 extension header, whether it is
 extension-header-only or a standalone protocol as well, and decide from that whether it
-inherits :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` alone or names additional bases.
+inherits :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` alone or names additional bases.
 
 And the limb that decides is **whether a primary source shows the header carried
 directly as an IPv4 payload**:
 
-*  :class:`~pcapkit.protocols.internet.ah.AH` -- :rfc:`4302#section-3.1.1`, *"In the
+*  :class:`~pcapkit.protocols.internet.ah.protocol.AH` -- :rfc:`4302#section-3.1.1`, *"In the
    context of IPv4, this calls for placing AH after the IP header"*, with a
    before-and-after IPv4 diagram.
-*  :class:`~pcapkit.protocols.internet.esp.ESP` -- :rfc:`4303#section-3.1.1`, the
+*  :class:`~pcapkit.protocols.internet.esp.protocol.ESP` -- :rfc:`4303#section-3.1.1`, the
    same sentence and the same diagram for ESP.
-*  :class:`~pcapkit.protocols.internet.hip.HIP` -- :rfc:`7401#appendix-C.2`,
+*  :class:`~pcapkit.protocols.internet.hip.protocol.HIP` -- :rfc:`7401#appendix-C.2`,
    *"IPv4 HIP Packet (I1 Packet)"*, whose worked checksum is over an IPv4 header
    carrying ``Next Header: 139``. :rfc:`7401#section-5.1` also calls the HIP header
    *"logically an IPv6 extension header"*, so HIP is genuinely both.
 
-:class:`~pcapkit.protocols.internet.mh.MH` is the instructive failure, because it
+:class:`~pcapkit.protocols.internet.mh.protocol.MH` is the instructive failure, because it
 **is** a protocol in its own right and still does not qualify:
 :rfc:`6275#section-6.1.1` defines its checksum over a pseudo-header of *"IPv6 header
 fields"* whose addresses are *"the addresses that appear in the Source and
@@ -105,7 +105,7 @@ is no IPv4 variant of that computation -- and the IPv4 equivalent function is no
 protocol 135 at all, since :rfc:`5944` carries Mobile IPv4 over UDP port 434.
 ``Shim6`` (140) has the same shape and the same verdict; this package carries no parser
 class for it, so nothing implements the classification, but a future one inherits
-:class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` alone.
+:class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` alone.
 
 .. important::
 
@@ -115,13 +115,13 @@ class for it, so nothing implements the classification, but a future one inherit
    review of :issue:`917` and not
    taken, so MH and ``Shim6`` stay extension-only. A header that is a protocol in its
    own right but structurally cannot be an IPv4 payload names
-   :class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` alone.
+   :class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` alone.
 
 Explicit Base Declarations
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Name the second base **explicitly**, even where it is already in the MRO.
-:class:`~pcapkit.protocols.internet.ipv6_ext.IPv6_Ext` derives
+:class:`~pcapkit.protocols.internet.ipv6_ext.protocol.IPv6_Ext` derives
 :class:`~pcapkit.protocols.internet.internet.Internet`, so every extension header
 reaches :class:`~pcapkit.protocols.internet.internet.Internet` transitively and an
 ``__mro__`` check cannot tell the two groups apart. The declaration is the only place the
@@ -140,7 +140,7 @@ held to the convention whether or not anyone remembers this page.
 ESP's Two Facts
 ~~~~~~~~~~~~~~~
 
-Two facts about :class:`~pcapkit.protocols.internet.esp.ESP` coexist, and each is
+Two facts about :class:`~pcapkit.protocols.internet.esp.protocol.ESP` coexist, and each is
 routinely mistaken for a refutation of the other.
 
 **It is an extension header.** IANA's *IPv6 Extension Header Types* registry lists
@@ -157,14 +157,14 @@ carries the same extension-mode contract as its siblings.
 **And it terminates the chain walk.** :rfc:`4303` places ESP's Next Header byte
 inside the *encrypted* trailer, so with no key material there is nothing to continue
 on: ESP's own data model reports ``next`` as :obj:`None`, and
-:meth:`IPv6._decode_next_layer <pcapkit.protocols.internet.ipv6.IPv6._decode_next_layer>`
+:meth:`IPv6._decode_next_layer <pcapkit.protocols.internet.ipv6.protocol.IPv6._decode_next_layer>`
 ends the ordinary way one iteration later -- not because ESP lacks a parser, which
 it has, but because it cannot hand the walk a successor.
 
 The full reasoning, including how ESP's terminal case differs from 253/254's, is
 written where the code is and is deliberately not restated at length here: see
-:meth:`IPv6._import_next_layer <pcapkit.protocols.internet.ipv6.IPv6._import_next_layer>`
-and the module docstring of :mod:`pcapkit.protocols.internet.ipv6_ext`.
+:meth:`IPv6._import_next_layer <pcapkit.protocols.internet.ipv6.protocol.IPv6._import_next_layer>`
+and the module docstring of :mod:`pcapkit.protocols.internet.ipv6_ext.protocol`.
 
 .. caution::
 

docs/source/contributing/conventions/extension-header-subclassing.rst

@@ -19,9 +19,9 @@ House Conventions
      :meth:`~pcapkit.protocols.protocol.Protocol.__index__` returns;
    * :ref:`extension-header-subclassing` -- a protocol class hierarchy;
    * :ref:`protocol-family-grouping` -- how a protocol family's base and members
-     spell :meth:`~pcapkit.protocols.protocol.Protocol.id`, how a protocol class
-     is named after its module, and how quietly a next-layer fallback keeps a
-     refused layer as :class:`~pcapkit.protocols.misc.raw.Raw`;
+     spell :meth:`~pcapkit.protocols.protocol.Protocol.id`, how a protocol's
+     package, modules and classes are named, and how quietly a next-layer fallback keeps a
+     refused layer as :class:`~pcapkit.protocols.misc.raw.protocol.Raw`;
    * :ref:`protocol-member-order` -- the order of members in a protocol class body;
    * :ref:`protocol-layer-placement` -- which subpackage a dissector belongs in;
    * :ref:`process` -- the repository rather than any of its code;

docs/source/contributing/conventions/index.rst

@@ -304,8 +304,8 @@ carrying it are that kind:
    :issue:`759` raising one where a
    single-bit lookup used to return a member;
 *  a public attribute's meaning --
-   :issue:`618` swapping :attr:`Frame.len <pcapkit.protocols.data.misc.pcap.frame.Frame.len>`
-   and :attr:`Frame.cap_len <pcapkit.protocols.data.misc.pcap.frame.Frame.cap_len>`
+   :issue:`618` swapping :attr:`Frame.len <pcapkit.protocols.misc.pcap.frame.data.Frame.len>`
+   and :attr:`Frame.cap_len <pcapkit.protocols.misc.pcap.frame.data.Frame.cap_len>`
    between the PCAP and PCAP-NG readers;
 *  a signature or a name a caller writes --
    :issue:`806` retyping