Skip to content

Commit 2198c11

Browse files
JarryShawJarryShaw
committed
feat(vendor): add the ARPHRD crawler, DATA-attribute shape (#2495)
Every pcapkit.const module needs a matching pcapkit.vendor crawler, with no exception (owner ruling on #2495) -- a hand-maintained registry uses the existing DATA-attribute pattern (pcapkit.vendor.ipx.packet/socket) instead of a live-URL scrape. - New pcapkit.vendor.linux.arphrd.ARPHRD crawler, DATA transcribed from <linux/if_arp.h>, same member/value/FLAG/_missing_ shape as the committed const module -- regenerating is a no-op against the tree. - Regenerate pcapkit.const.linux.arphrd from it; only the docstrings change to the standard templated wording. - Rewrite const-vendor-pairing.rst: drop the exception section, document the DATA-attribute pattern against its real precedent. - Bump test_vendor_missing_body_unit.py's crawler-with-miss-line count. Tests: 124 passed, 1 pre-existing skip, 1373 subtests, across the convention, vendor-miss-body, public-api and registry-protocol suites.
1 parent 2f6c333 commit 2198c11

6 files changed

Lines changed: 184 additions & 31 deletions

File tree

‎docs/source/contributing/conventions/const-vendor-pairing.rst‎

Lines changed: 36 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,41 @@ Const-Vendor Pairing
44
---------------------
55

66
**Every module under** :mod:`pcapkit.const` **is generated by a matching crawler
7-
under** :mod:`pcapkit.vendor`. A registry copied from an IANA, IETF or similar
8-
web-published table is never hand-maintained -- the crawler re-scrapes the source and
9-
regenerates the module, so a hand edit to the generated file is silently discarded on
10-
the next run. :issue:`2441`, raised in cross-review and knowingly deferred rather than
11-
fixed.
7+
under** :mod:`pcapkit.vendor`, with no exception. A registry copied from an IANA,
8+
IETF or similar web-published table is never hand-maintained -- the crawler
9+
re-scrapes the source and regenerates the module, so a hand edit to the generated
10+
file is silently discarded on the next run. :issue:`2441`, raised in cross-review
11+
and knowingly deferred rather than fixed.
1212

13-
The Exception
14-
~~~~~~~~~~~~~
13+
The ``DATA`` Attribute, for a Registry With No Web Source
14+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1515

16-
**A source that is not a web-crawlable table is the one legitimate reason to hand-write
17-
a const module.** :class:`~pcapkit.const.linux.arphrd.ARPHRD`'s source is the Linux
18-
kernel header ``<linux/if_arp.h>``, not an IANA page, and its docstring says so plainly
19-
-- *"Only the values that pcapkit names are listed"* -- rather than claiming full
20-
coverage from a registry :mod:`pcapkit.vendor` could crawl. The ``EnumRegistry`` shape
21-
it was moved to on :pr:`2443` is a structural decision for API uniformity with every
22-
other registry, not a claim that a crawler produces the file.
16+
A source that is not a web-crawlable table does not excuse a module from this
17+
pairing -- it changes what the crawler fetches. :class:`~pcapkit.vendor.default.Vendor`
18+
reads its registry two ways: :meth:`~pcapkit.vendor.default.Vendor._request` either
19+
``GET``\ s :attr:`~pcapkit.vendor.default.Vendor.LINK`, or, when a subclass overrides
20+
:meth:`~pcapkit.vendor.default.Vendor.request` to take no arguments, returns a
21+
module-level ``DATA`` literal instead and never touches the network.
22+
:mod:`pcapkit.vendor.ipx.packet` and :mod:`pcapkit.vendor.ipx.socket` are the
23+
precedent: both read a hand-transcribed ``DATA`` dict because their source,
24+
Novell's and Xerox's own IPX socket/packet-type tables, went offline (:issue:`507`,
25+
:issue:`518`).
26+
27+
:class:`~pcapkit.const.linux.arphrd.ARPHRD` follows this pattern: its source is the
28+
Linux kernel header ``<linux/if_arp.h>``, C source
29+
under the kernel's own versioning rather than an IANA page, so
30+
:mod:`pcapkit.vendor.linux.arphrd` reads a ``DATA`` dict of
31+
``{code: (name, description)}`` transcribed from the header instead of fetching a
32+
URL. The crawler's :meth:`~pcapkit.vendor.default.Vendor.process` renders that dict
33+
into the same enumeration members, the same ``FLAG`` bound and the same
34+
``_missing_`` body the committed :mod:`pcapkit.const.linux.arphrd` carries --
35+
running it is a no-op against the file in the tree, exactly as a web-scraped
36+
crawler's run is. The explanatory prose that once lived in the hand-maintained
37+
``const`` module -- which values are listed and why, the cross-reference to
38+
:class:`~pcapkit.const.arp.hardware.Hardware` -- now lives in the crawler's own
39+
module docstring and comments, alongside the ``DATA`` literal it documents; the
40+
generated ``const`` module carries only the plain, templated docstring every
41+
other generated module does.
2342

2443
Known Violation
2544
~~~~~~~~~~~~~~~
@@ -30,4 +49,6 @@ IANA registry and claim to name every value IANA assigns, with no
3049
:mod:`pcapkit.vendor` crawler behind either one -- no ``pcapkit/vendor/tls/``
3150
directory exists, and neither class appears in :data:`pcapkit.vendor.__all__`. This
3251
was flagged in that PR's own review and merged anyway on the stated reasoning that the
33-
crawler would follow later. Tracked as :issue:`2495`.
52+
crawler would follow later. Tracked as :issue:`2495`. Unlike ARPHRD, TLS's source
53+
*is* a live, web-published IANA table, so its eventual crawler takes the normal
54+
``LINK``-and-scrape shape, not the ``DATA`` attribute.

‎pcapkit/const/linux/arphrd.py‎

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,12 @@
11
# -*- coding: utf-8 -*-
2-
"""Linux ARP Hardware Types
2+
# pylint: disable=line-too-long,consider-using-f-string
3+
"""Linux ARP hardware types
34
==============================
45
56
.. module:: pcapkit.const.linux.arphrd
67
7-
This module contains the constant enumeration for the Linux ``ARPHRD_*``
8-
hardware types of ``<linux/if_arp.h>``, the ``arphrd_type`` of the Linux
9-
"cooked" capture encapsulations. Only the values that pcapkit names are listed.
10-
The rest of the numbering is IANA's, see
11-
:class:`~pcapkit.const.arp.hardware.Hardware`, which differs for most
12-
non-Ethernet types.
8+
This module contains the constant enumeration for **Linux ARP hardware types**,
9+
which is automatically generated from :class:`pcapkit.vendor.linux.arphrd.ARPHRD`.
1310
1411
"""
1512

@@ -49,29 +46,33 @@ class ARPHRD(EnumRegistry, IntEnum):
4946

5047
@classmethod
5148
def _validate_value(cls, value: 'Any') -> 'None':
52-
"""Reject ``value`` unless it is an ``arphrd_type`` (a 16-bit value).
49+
"""Reject ``value`` unless it passes the range check :meth:`_missing_` makes.
5350
5451
Overrides :meth:`~pcapkit.corekit.enum.EnumLookup._validate_value`
55-
so that :meth:`~pcapkit.corekit.enum.EnumRegistry.register` refuses
56-
the values :meth:`_missing_` refuses.
52+
with that check and the message it raises, so
53+
:meth:`~pcapkit.corekit.enum.EnumRegistry.register` refuses exactly the
54+
values :meth:`~pcapkit.corekit.enum.EnumLookup.get` refuses, in the
55+
same words (:issue:`1816`).
56+
57+
Args:
58+
value: Value about to be looked up or minted.
5759
5860
Raises:
59-
EnumValueError: If ``value`` is not an integer in ``0..0xFFFF``.
61+
EnumValueError: If ``value`` fails that check. A
62+
:exc:`ValueError`, so an ``except ValueError`` caller is
63+
unaffected.
6064
6165
"""
6266
if not (isinstance(value, int) and 0x0000 <= value <= 0xFFFF):
6367
raise EnumValueError('%r is not a valid %s' % (value, cls.__name__))
6468

6569
@classmethod
6670
def _missing_(cls, value: 'int') -> 'ARPHRD':
67-
"""Resolve an unnamed ``arphrd_type`` to an unregistered member.
71+
"""Lookup function used when value is not found.
6872
6973
Args:
7074
value: Value to get enum item.
7175
72-
Raises:
73-
ValueError: If ``value`` is not a 16-bit integer.
74-
7576
"""
7677
if not (isinstance(value, int) and 0x0000 <= value <= 0xFFFF):
7778
raise ValueError('%r is not a valid %s' % (value, cls.__name__))

‎pcapkit/vendor/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,7 @@ def __getattr__(name: 'str') -> 'typing.NoReturn':
114114
from pcapkit.vendor.ipv6 import *
115115
from pcapkit.vendor.ipx import *
116116
from pcapkit.vendor.l2tp import *
117+
from pcapkit.vendor.linux import *
117118
from pcapkit.vendor.mh import *
118119
from pcapkit.vendor.ngap import *
119120
from pcapkit.vendor.ospf import *
@@ -153,6 +154,8 @@ def __getattr__(name: 'str') -> 'typing.NoReturn':
153154
'IPX_Packet', 'IPX_Socket',
154155
# L2TP
155156
'L2TP_Type',
157+
# Linux
158+
'Linux_ARPHRD',
156159
# MH
157160
'MH_Packet', 'MH_Option', 'MH_DNSStatusCode', 'MH_ACKStatusCode',
158161
'MH_MNIDSubtype', 'MH_StatusCode', 'MH_EnumeratingAlgorithm',

‎pcapkit/vendor/linux/__init__.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# -*- coding: utf-8 -*-
2+
# pylint: disable=unused-import
3+
""":mod:`pcapkit.const.linux` Vendor Crawlers
4+
===============================================
5+
6+
.. module:: pcapkit.vendor.linux
7+
8+
This module contains all vendor crawlers of the Linux-specific encapsulations,
9+
e.g. :class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`. Available crawlers
10+
include:
11+
12+
.. list-table::
13+
14+
* - :class:`Linux_ARPHRD <pcapkit.vendor.linux.arphrd.ARPHRD>`
15+
- Linux ARP Hardware Types
16+
17+
"""
18+
19+
from pcapkit.vendor.linux.arphrd import ARPHRD as Linux_ARPHRD
20+
21+
__all__ = ['Linux_ARPHRD']

‎pcapkit/vendor/linux/arphrd.py‎

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# -*- coding: utf-8 -*-
2+
"""Linux ARP Hardware Types
3+
==============================
4+
5+
.. module:: pcapkit.vendor.linux.arphrd
6+
7+
This module contains the vendor crawler for **Linux ARP Hardware Types**,
8+
which is automatically generating :class:`pcapkit.const.linux.arphrd.ARPHRD`.
9+
10+
"""
11+
12+
import collections
13+
import sys
14+
from typing import TYPE_CHECKING
15+
16+
from pcapkit.vendor.default import Vendor
17+
18+
if TYPE_CHECKING:
19+
from collections import Counter
20+
21+
__all__ = ['ARPHRD']
22+
23+
###############################################################################
24+
# NOTE: this crawler does not crawl. Its source is the Linux kernel header
25+
# ``<linux/if_arp.h>``, which assigns ``ARPHRD_*`` values in C source under the
26+
# kernel's own versioning, not an IANA or IETF table :mod:`pcapkit.vendor` could
27+
# scrape. Per the const-vendor-pairing convention, a hand-maintained registry
28+
# still gets a crawler, reading :data:`~pcapkit.vendor.linux.arphrd.DATA`
29+
# instead of fetching a URL -- the same shape as
30+
# :mod:`pcapkit.vendor.ipx.packet` and :mod:`pcapkit.vendor.ipx.socket`.
31+
#
32+
# pcapkit names only the ``ARPHRD_*`` values its Linux "cooked" capture
33+
# dissector (:class:`~pcapkit.protocols.link.linux_sll.LinuxSLL`) actually
34+
# matches on, not every value the header defines. The rest of the ARP
35+
# hardware-type numbering is IANA's own
36+
# (:class:`~pcapkit.const.arp.hardware.Hardware`), which differs for most
37+
# non-Ethernet types.
38+
###############################################################################
39+
40+
#: ARPHRD hardware-type registry, transcribed from ``<linux/if_arp.h>`` for the
41+
#: values pcapkit's Linux "cooked" capture dissector names.
42+
#:
43+
#: Maps an ``arphrd_type`` to the enumeration name it takes and the comment
44+
#: rendered above it. Names go through :meth:`~Vendor.rename`, so the generated
45+
#: member is the :meth:`~Vendor.safe_name` of the first element.
46+
DATA = {
47+
0: ('NETROM', 'NET/ROM pseudo.'),
48+
770: ('FRAD', 'Frame Relay Access Device'),
49+
778: ('IPGRE', 'GRE over IP'),
50+
803: ('IEEE80211_RADIOTAP', 'IEEE 802.11 with radiotap header'),
51+
823: ('IP6GRE', 'GRE over IPv6'),
52+
824: ('NETLINK', 'Kernel Netlink socket'),
53+
} # type: dict[int, tuple[str, str]]
54+
55+
56+
class ARPHRD(Vendor):
57+
"""Linux ARP hardware types [``<linux/if_arp.h>``]"""
58+
59+
#: Value limit checker.
60+
FLAG = 'isinstance(value, int) and 0x0000 <= value <= 0xFFFF'
61+
62+
def request(self) -> 'dict[int, tuple[str, str]]': # type: ignore[override] # pylint: disable=arguments-differ
63+
"""Fetch registry data.
64+
65+
Returns:
66+
Registry data (:data:`~pcapkit.vendor.linux.arphrd.DATA`).
67+
68+
"""
69+
return DATA
70+
71+
def count(self, data: 'dict[int, tuple[str, str]]') -> 'Counter[str]': # type: ignore[override]
72+
"""Count field records.
73+
74+
Args:
75+
data: Registry data.
76+
77+
Returns:
78+
Field recordings.
79+
80+
"""
81+
return collections.Counter(self.safe_name(name) for name, _ in data.values())
82+
83+
def process(self, data: 'dict[int, tuple[str, str]]') -> 'tuple[list[str], list[str]]': # type: ignore[override]
84+
"""Process registry data.
85+
86+
Args:
87+
data: Registry data.
88+
89+
Returns:
90+
Enumeration fields and missing fields.
91+
92+
"""
93+
enum = [] # type: list[str]
94+
miss = [
95+
"return cls._unregistered_member(value, 'Unassigned')",
96+
]
97+
98+
for code, (name, desc) in data.items():
99+
pval = str(code)
100+
renm = self.rename(name, pval)
101+
102+
enum.append(f'#: {self.wrap_comment(desc)}\n {renm} = {pval}')
103+
return enum, miss
104+
105+
106+
if __name__ == '__main__':
107+
sys.exit(ARPHRD()) # type: ignore[arg-type]

‎tests/vendor/test_vendor_missing_body_unit.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@
8585
#: or under a different variable name entirely is skipped by construction too,
8686
#: same as one built by ``.append()``. Those are covered behaviourally, not
8787
#: statically, by :file:`tests/const/test_const_enum_lookup.py`.
88-
EXPECTED_CRAWLERS_WITH_CONSTANT_MISS_LINES = 35
88+
EXPECTED_CRAWLERS_WITH_CONSTANT_MISS_LINES = 36
8989

9090

9191
def _constant_miss_lines() -> 'dict[str, list[str]]':

0 commit comments

Comments
 (0)