Skip to content

Commit 9fb7ad0

Browse files
committed
refactor(corekit): merge the two DeferredPacket classes into corekit.packet (#1516)
- pcapkit.corekit.packet.DeferredPacket replaces the copies in reassembly/data/data.py and traceflow/data/data.py. Their method bodies were AST-identical; the one difference, which module's Deferred the isinstance named, is now the __deferred__ class attribute each subclass sets, so the module imports nothing from foundation or protocols. - __init_subclass__ refuses a subclass that does not list 'packet' in __additional__ or name a class in __deferred__, either of which would otherwise fail silently on the first read. - ip.Datagram, tcp.Datagram and traceflow Index import it from corekit and set __deferred__ to their own Deferred, which stays where it was. - DeferredPacket stays importable from its three old public paths, now re-exported from corekit.packet. New docs page corekit/packet.rst; the two foundation pages point to it instead of documenting a copy. - tests/corekit/test_deferred_packet_1516_unit.py pins laziness, the hook, the guard, the import layering, the three models' wiring and the old paths. Refs #1516
1 parent 16fbea0 commit 9fb7ad0

11 files changed

Lines changed: 509 additions & 173 deletions

File tree

‎docs/source/pcapkit/corekit/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ inherits from.
2525
io
2626
module
2727
multidict
28+
packet
2829
protochain
2930
sentinels
3031
version
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
Deferred Packet
2+
===============
3+
4+
.. module:: pcapkit.corekit.packet
5+
6+
:mod:`pcapkit.corekit.packet` contains
7+
:class:`~pcapkit.corekit.packet.DeferredPacket`, the mixin that lets an
8+
:class:`~pcapkit.corekit.infoclass.Info` subclass hold its ``packet`` field as
9+
a placeholder and resolve it on first read. It is shared by the reassembled
10+
datagrams of :mod:`pcapkit.foundation.reassembly` and the traced flows of
11+
:mod:`pcapkit.foundation.traceflow`, whose ``packet`` is a second parse of the
12+
payload and a reassembly of the stream respectively.
13+
14+
A subclass is declared ``class X(DeferredPacket, Info)``, lists ``packet`` in
15+
its ``__additional__``, and names its placeholder class in
16+
:attr:`~pcapkit.corekit.packet.DeferredPacket.__deferred__`;
17+
:meth:`~pcapkit.corekit.packet.DeferredPacket.__init_subclass__` refuses one
18+
that does not. Listing ``packet`` in ``__additional__`` is what makes the field
19+
lazy: :class:`~pcapkit.corekit.infoclass.Info` then stores it under a mangled
20+
key, so reading it reaches ``__getattr__``, while ``dict(x)``,
21+
:meth:`~pcapkit.corekit.packet.DeferredPacket.to_dict` and iteration still
22+
report it as ``packet``.
23+
24+
Resolution happens **at most once**: the placeholder is called with no
25+
arguments, and its result replaces it, so every later read returns that same
26+
object.
27+
28+
.. autoclass:: pcapkit.corekit.packet.DeferredPacket
29+
:no-members:
30+
:show-inheritance:
31+
32+
.. attribute:: __deferred__
33+
34+
Placeholder class the ``packet`` field may hold, set by each subclass. A
35+
value of this class is resolved by calling it with no arguments; an
36+
instance of any other class, :data:`None` included, is not.
37+
38+
.. automethod:: __init_subclass__
39+
.. automethod:: __analyse__
40+
.. automethod:: to_dict

‎docs/source/pcapkit/foundation/reassembly/index.rst‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,4 @@ Auxiliary Data
6565
:members:
6666
:show-inheritance:
6767

68-
.. autoclass:: pcapkit.foundation.reassembly.data.data.DeferredPacket
69-
:members:
70-
:show-inheritance:
68+
``DeferredPacket`` is re-exported here from :mod:`pcapkit.corekit.packet`; see :doc:`/pcapkit/corekit/packet`.

‎docs/source/pcapkit/foundation/traceflow/index.rst‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,4 @@ Auxiliary Data
6464
:members:
6565
:show-inheritance:
6666

67-
.. autoclass:: pcapkit.foundation.traceflow.data.data.DeferredPacket
68-
:members:
69-
:show-inheritance:
67+
``DeferredPacket`` is re-exported here from :mod:`pcapkit.corekit.packet`; see :doc:`/pcapkit/corekit/packet`.

‎pcapkit/corekit/packet.py‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
# -*- coding: utf-8 -*-
2+
"""Deferred Packet
3+
===================
4+
5+
.. module:: pcapkit.corekit.packet
6+
7+
:mod:`pcapkit.corekit.packet` contains
8+
:class:`~pcapkit.corekit.packet.DeferredPacket`, the mixin that lets an
9+
:class:`~pcapkit.corekit.infoclass.Info` subclass hold its ``packet`` field as
10+
a placeholder and resolve it on first read. A reassembled datagram's ``packet``
11+
is a second parse of its payload, and a traced flow's is a reassembly of its
12+
stream; most callers want neither, so both are postponed until somebody reads
13+
them.
14+
15+
A subclass keeps a three-part contract, which
16+
:meth:`DeferredPacket.__init_subclass__` checks at class creation:
17+
18+
* It is declared ``class X(DeferredPacket, Info)``, mixin first, so the methods
19+
here take precedence over :class:`~pcapkit.corekit.infoclass.Info`'s own.
20+
* It lists ``packet`` in its ``__additional__``. That is what makes the field
21+
lazy at all: :class:`~pcapkit.corekit.infoclass.Info` stores a field named
22+
there under a mangled key and maps it back on the way out, so ``packet``
23+
never lands in :attr:`~object.__dict__` under its own name. Reading it
24+
therefore reaches :meth:`DeferredPacket.__getattr__`, where the placeholder
25+
can be resolved, while ``dict(x)``, :meth:`DeferredPacket.to_dict` and
26+
iteration still report the field as ``packet``.
27+
* It names its placeholder class in
28+
:attr:`~DeferredPacket.__deferred__`. A value of that class is called with no
29+
arguments to produce the real ``packet``; any other value already is one.
30+
31+
Resolution happens **at most once**: the result replaces the placeholder in
32+
:attr:`~object.__dict__`, so every later read returns that same object and the
33+
placeholder is never called again.
34+
35+
This module imports nothing from :mod:`pcapkit.foundation` or
36+
:mod:`pcapkit.protocols`. Each placeholder class stays with the subsystem that
37+
creates it, and only the subclass knows which one it holds.
38+
39+
"""
40+
from typing import TYPE_CHECKING
41+
42+
from pcapkit.utilities.exceptions import InfoError
43+
44+
if TYPE_CHECKING:
45+
from typing import Any, Callable, ClassVar, Type
46+
47+
__all__ = ['DeferredPacket']
48+
49+
50+
class DeferredPacket:
51+
"""Resolves a deferred ``packet`` field on first read.
52+
53+
A mixin for :class:`~pcapkit.corekit.infoclass.Info` subclasses; see
54+
:mod:`pcapkit.corekit.packet` for the contract a subclass keeps. Reading
55+
``packet`` -- as an attribute, as ``x['packet']``, or through
56+
:meth:`to_dict`, :func:`str` or :func:`repr` -- resolves a placeholder once
57+
and keeps the result. ``'packet' in x``, :func:`len` and iteration report the
58+
field by name without resolving it.
59+
60+
It is shared by
61+
:class:`pcapkit.foundation.reassembly.data.ip.Datagram`,
62+
:class:`pcapkit.foundation.reassembly.data.tcp.Datagram` and
63+
:class:`pcapkit.foundation.traceflow.data.tcp.Index`, which differ only in
64+
the placeholder class each names in :attr:`__deferred__`
65+
(:issue:`1516`). It is still importable from where it lived before:
66+
:mod:`pcapkit.foundation.reassembly.data`,
67+
:mod:`pcapkit.foundation.reassembly.data.data` and
68+
:mod:`pcapkit.foundation.traceflow.data.data`.
69+
70+
"""
71+
72+
if TYPE_CHECKING:
73+
#: Placeholder class the ``packet`` field may hold, set by each subclass.
74+
#: A value of this class is resolved by calling it with no arguments;
75+
#: an instance of any other class, :data:`None` included, is not.
76+
__deferred__: 'ClassVar[Type[Callable[[], Any]]]'
77+
78+
# NOTE: the ``super()`` calls below are suppressed for both checkers. They are
79+
# undefined *on this mixin*, which is what a mixin is -- the base arrives at
80+
# the point of use, where every subclass is declared
81+
# ``class X(DeferredPacket, Info)`` and :class:`~pcapkit.corekit.infoclass.Info`
82+
# supplies all three. Neither mypy nor pylint can see that from here, and
83+
# pylint calls it an *error* rather than a warning.
84+
85+
def __init_subclass__(cls, /, *args: 'Any', **kwargs: 'Any') -> 'None':
86+
"""Refuse a subclass that does not keep the contract.
87+
88+
Either omission would otherwise fail silently, and only on a read.
89+
Without ``packet`` in ``__additional__`` the field lands in
90+
:attr:`~object.__dict__` under its own name, :meth:`__getattr__` is never
91+
reached, and the placeholder itself comes back. Without
92+
:attr:`__deferred__`, :meth:`__analyse__` raises :exc:`AttributeError`
93+
from inside :meth:`__getattr__`, which ``getattr(x, 'packet', None)``
94+
takes for a missing attribute and answers with :data:`None`.
95+
96+
Args:
97+
*args: Arbitrary positional arguments.
98+
**kwargs: Arbitrary keyword arguments in class definition.
99+
100+
Raises:
101+
InfoError: If ``cls`` does not list ``packet`` in its
102+
``__additional__``, or does not name a class in its
103+
:attr:`__deferred__`.
104+
105+
"""
106+
super().__init_subclass__(*args, **kwargs)
107+
108+
if 'packet' not in getattr(cls, '__additional__', ()):
109+
raise InfoError(f"{cls.__name__}: 'packet' is not listed in __additional__, "
110+
'so reading it would never resolve the placeholder')
111+
if not isinstance(getattr(cls, '__deferred__', None), type):
112+
raise InfoError(f"{cls.__name__}: __deferred__ does not name the placeholder class "
113+
"that 'packet' may hold")
114+
115+
def __analyse__(self) -> 'Any':
116+
"""Resolve the ``packet`` field, at most once.
117+
118+
A placeholder, i.e. an instance of :attr:`__deferred__`, is called with
119+
no arguments and its result replaces it in :attr:`~object.__dict__`. Any
120+
other value is returned as it is.
121+
122+
Returns:
123+
The resolved ``packet``, which may be :data:`None` when the subclass
124+
stored that instead of a placeholder.
125+
126+
"""
127+
key = self.__map__.get('packet', 'packet')
128+
value = self.__dict__[key]
129+
if isinstance(value, self.__deferred__):
130+
value = value()
131+
self.__dict__[key] = value
132+
return value
133+
134+
def __getattr__(self, name: 'str') -> 'Any':
135+
# NOTE: reached only for names absent from ``__dict__``, which ``packet``
136+
# always is -- see ``__additional__`` in the module docstring. Everything
137+
# else has to raise, or a typo would silently answer with the resolved
138+
# ``packet``.
139+
if name != 'packet':
140+
raise AttributeError(f'{type(self).__name__!r} object has no attribute {name!r}')
141+
return self.__analyse__()
142+
143+
def __getitem__(self, name: 'str') -> 'Any':
144+
if name == 'packet':
145+
return self.__analyse__()
146+
return super().__getitem__(name) # type: ignore[misc] # pylint: disable=no-member
147+
148+
def __contains__(self, name: 'object') -> 'bool':
149+
# NOTE: ``Mapping.__contains__`` answers by fetching the value, which
150+
# would resolve the placeholder merely to decide that the field exists.
151+
# ``packet`` is a declared field, so it is always there.
152+
return name == 'packet' or super().__contains__(name) # type: ignore[misc] # pylint: disable=no-member
153+
154+
def __str__(self) -> 'str':
155+
self.__analyse__()
156+
return super().__str__()
157+
158+
def __repr__(self) -> 'str':
159+
self.__analyse__()
160+
return super().__repr__()
161+
162+
def to_dict(self) -> 'dict[str, Any]':
163+
"""Convert the instance into :obj:`dict`.
164+
165+
Returns:
166+
The fields, with ``packet`` resolved if it had not been read yet --
167+
a :obj:`dict` holding the placeholder would leak an implementation
168+
detail into what is meant to be plain data.
169+
170+
"""
171+
self.__analyse__()
172+
return super().to_dict() # type: ignore[misc] # pylint: disable=no-member

‎pcapkit/foundation/reassembly/data/data.py‎

Lines changed: 5 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@
55

66
from pcapkit.corekit.enum import EnumLookup
77
from pcapkit.corekit.infoclass import Info, info_final
8+
from pcapkit.corekit.packet import DeferredPacket
89
from pcapkit.utilities.compat import StrEnum, auto
910

1011
__all__ = ['ReassemblyData', 'Completion', 'Deferred', 'DeferredPacket']
1112

1213
if TYPE_CHECKING:
13-
from typing import Any, Callable, Optional
14+
from typing import Callable, Optional
1415

1516
from pcapkit.const.reg.transtype import TransType
1617
from pcapkit.foundation.reassembly.data.ip import Datagram as IP_Datagram
@@ -124,9 +125,9 @@ class Deferred:
124125
:file:`http.pcap` pass against 1117.
125126
126127
Holding the call here defers it to the first read of
127-
:attr:`Datagram.packet`, so a caller that wants the parsed payload still gets
128-
exactly the object an eager call would produce, and one that does not never
129-
pays for it.
128+
:attr:`Datagram.packet`, where :class:`~pcapkit.corekit.packet.DeferredPacket`
129+
runs it, so a caller that wants the parsed payload still gets exactly the
130+
object an eager call would produce, and one that does not never pays for it.
130131
131132
Args:
132133
analyze: The analyser to call, i.e.
@@ -155,86 +156,6 @@ def __call__(self) -> 'ProtocolBase':
155156
return self.analyze(self.proto, self.payload)
156157

157158

158-
class DeferredPacket:
159-
"""Resolves a :class:`Deferred` ``packet`` field on first read.
160-
161-
A reassembled datagram's ``packet`` is the parsed form of the payload it just
162-
reassembled, and both reassemblers can hand a :class:`Deferred` in its place.
163-
This carries the reading half of that arrangement, so the two ``Datagram``
164-
models share it rather than each declaring it.
165-
166-
A subclass has to list ``packet`` in its ``__additional__``. That is what makes
167-
the field lazy at all: :class:`~pcapkit.corekit.infoclass.Info` stores a field
168-
whose name is a *builtin* name under a mangled key and maps it back on the way
169-
out, so ``packet`` never lands in :attr:`~object.__dict__` itself -- which
170-
routes reading it through :meth:`__getattr__`, where the deferred analysis can
171-
run, while ``dict(datagram)``, :meth:`to_dict` and iteration still report the
172-
field under its own name.
173-
174-
"""
175-
176-
# NOTE: the ``super()`` calls below are suppressed for both checkers. They are
177-
# undefined *on this mixin*, which is what a mixin is -- the base arrives at
178-
# the point of use, where every subclass is declared
179-
# ``class X(DeferredPacket, Info)`` and :class:`~pcapkit.corekit.infoclass.Info`
180-
# supplies all three. Neither mypy nor pylint can see that from here, and
181-
# pylint calls it an *error* rather than a warning.
182-
183-
def __analyse__(self) -> 'Optional[ProtocolBase]':
184-
"""Resolve a deferred analysis, at most once.
185-
186-
Returns:
187-
Parsed payload, or :data:`None` for a datagram reported as the tuple
188-
of its received runs.
189-
190-
"""
191-
key = self.__map__.get('packet', 'packet')
192-
value = self.__dict__[key]
193-
if isinstance(value, Deferred):
194-
value = value()
195-
self.__dict__[key] = value
196-
return value
197-
198-
def __getattr__(self, name: 'str') -> 'Any':
199-
# NOTE: reached only for names absent from ``__dict__``, which ``packet``
200-
# always is -- see ``__additional__`` above. Everything else has to raise,
201-
# or a typo would silently answer with a parsed payload.
202-
if name != 'packet':
203-
raise AttributeError(f'{type(self).__name__!r} object has no attribute {name!r}')
204-
return self.__analyse__()
205-
206-
def __getitem__(self, name: 'str') -> 'Any':
207-
if name == 'packet':
208-
return self.__analyse__()
209-
return super().__getitem__(name) # type: ignore[misc] # pylint: disable=no-member
210-
211-
def __contains__(self, name: 'object') -> 'bool':
212-
# NOTE: ``Mapping.__contains__`` answers by fetching the value, which
213-
# would run the deferred analysis merely to decide that the field exists.
214-
# ``packet`` is a declared field, so it is always there.
215-
return name == 'packet' or super().__contains__(name) # type: ignore[misc] # pylint: disable=no-member
216-
217-
def __str__(self) -> 'str':
218-
self.__analyse__()
219-
return super().__str__()
220-
221-
def __repr__(self) -> 'str':
222-
self.__analyse__()
223-
return super().__repr__()
224-
225-
def to_dict(self) -> 'dict[str, Any]':
226-
"""Convert :class:`Datagram` into :obj:`dict`.
227-
228-
Returns:
229-
The datagram's fields, with ``packet`` analysed if it had not been
230-
read yet -- a :obj:`dict` holding a :class:`Deferred` would leak an
231-
implementation detail into what is meant to be plain data.
232-
233-
"""
234-
self.__analyse__()
235-
return super().to_dict() # type: ignore[misc] # pylint: disable=no-member
236-
237-
238159
@info_final
239160
class ReassemblyData(Info):
240161
"""Data storage for reassembly."""

‎pcapkit/foundation/reassembly/data/ip.py‎

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44
from typing import TYPE_CHECKING, Generic, TypeVar
55

66
from pcapkit.corekit.infoclass import Info, info_final
7-
from pcapkit.foundation.reassembly.data.data import Completion, Deferred, DeferredPacket
7+
from pcapkit.corekit.packet import DeferredPacket
8+
from pcapkit.foundation.reassembly.data.data import Completion, Deferred
89
from pcapkit.utilities.compat import Tuple
910

1011
__all__ = [
@@ -86,7 +87,11 @@ class Datagram(DeferredPacket, Info, Generic[_AT]):
8687
#: itself -- which routes reading it through :meth:`__getattr__`, where a
8788
#: :class:`Deferred` analysis can be run, while ``dict(datagram)``,
8889
#: :meth:`to_dict` and iteration still report the field under its own name.
90+
#: See :class:`~pcapkit.corekit.packet.DeferredPacket`.
8991
__additional__ = ['packet']
92+
#: Placeholder class :attr:`packet` may hold, for
93+
#: :class:`~pcapkit.corekit.packet.DeferredPacket` to resolve.
94+
__deferred__ = Deferred
9095

9196
#: How completely the datagram was reassembled, and why reassembly stopped.
9297
#: Only :attr:`Completion.COMPLETE` is truthy, so ``if datagram.completed:``

‎pcapkit/foundation/reassembly/data/tcp.py‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44
from typing import TYPE_CHECKING, Generic, TypeVar
55

66
from pcapkit.corekit.infoclass import Info, info_final
7-
from pcapkit.foundation.reassembly.data.data import Completion, Deferred, DeferredPacket
7+
from pcapkit.corekit.packet import DeferredPacket
8+
from pcapkit.foundation.reassembly.data.data import Completion, Deferred
89
from pcapkit.utilities.compat import Tuple
910

1011
__all__ = [
@@ -92,8 +93,11 @@ class Datagram(DeferredPacket, Info, Generic[_AT]):
9293
#: as it would for a :obj:`bool`.
9394
completed: 'Completion'
9495
#: Listing ``packet`` here is what makes it lazy -- see
95-
#: :class:`~pcapkit.foundation.reassembly.data.data.DeferredPacket`.
96+
#: :class:`~pcapkit.corekit.packet.DeferredPacket`.
9697
__additional__ = ['packet']
98+
#: Placeholder class :attr:`packet` may hold, for
99+
#: :class:`~pcapkit.corekit.packet.DeferredPacket` to resolve.
100+
__deferred__ = Deferred
97101

98102
#: Original packet identifier.
99103
id: 'DatagramID[_AT]'

0 commit comments

Comments
 (0)