fix(pcap): read Header._read_protos in the file's byte order (#2312) (#2334)

Problem: Header._read_protos unpacked with lilendian=True whatever the
file's byte order, so on a big-endian file 00 00 00 01 read as
LinkType.Unassigned (16777216) rather than ETHERNET.

Fix: read in self._byte, falling back to little-endian before read()
has set it. The docstring says what it reads; the 1.5.0 changelog
entry for #1125 is extended and CHANGELOG.md regenerated.

Test: test_header_magic_protos_unit gains a big/little x size 1/2/4
case and a parsed big-endian header case; both fail on origin/main.

Author: JarryShaw

CHANGELOG.md

@@ -281,7 +281,7 @@ This resolves [#548](https://github.com/JarryShaw/PyPCAPKit/issues/548), which r
 - `IPv6._read_ip_hextet` sliced the traffic class from `hex[0:2]` and the flow label from `hex[2:]`, both including the version nibble, so `6abcdef0` read as `(6, 0x6a, 0xbcdef0)`; it now reads `(6, 0xab, 0xcdef0)`. The parser reads these fields through the schema, so only a direct call was affected ([#1094](https://github.com/JarryShaw/PyPCAPKit/issues/1094)).
 - the Mobility Header Message ID option wrote a zero fraction for every timestamp, and `MesgIDOption.post_process()` ignored the fraction on read. `_make_opt_mesg_id` now writes `round(frac * 2**32)`, clamped to 32 bits, and the reader adds `fraction / 2**32` seconds, so `timestamp` keeps sub-second precision ([#1093](https://github.com/JarryShaw/PyPCAPKit/issues/1093)).
 - HIP parameter defects. `HOST_ID` packed DI-Type and DI-Length as a 4-octet `BitField` where [RFC 7401](https://datatracker.ietf.org/doc/html/rfc7401) section 5.2.9 has one 16-bit word, so every built record carried two surplus octets and read back the wrong algorithm ([#1118](https://github.com/JarryShaw/PyPCAPKit/issues/1118)); with it, `make_samples.py` builds every HIP parameter. `HIP_CIPHER` warned at six Cipher IDs, which section 5.2.8 allows; it now warns only above six and keeps the first six, as a recipient must. The maker is unchanged ([#1092](https://github.com/JarryShaw/PyPCAPKit/issues/1092)). And the HIPv2-only `DH_GROUP_LIST`, `HIP_CIPHER`, `HIT_SUITE_LIST` and `TRANSPORT_FORMAT_LIST` were built and parsed in a HIPv1 packet; at version 1 the reader and the maker now raise `ProtocolError`, as `R1_Counter` and `HIP_TRANSFORM` already do on a version mismatch ([#1133](https://github.com/JarryShaw/PyPCAPKit/issues/1133)).
-- the PCAP `Header` could not rebuild itself. `_make_magic`'s explicit-endian branch ignored `nanosecond` ([#1096](https://github.com/JarryShaw/PyPCAPKit/issues/1096)), and `_make_data` returned a `magic_number` that `make()` does not take -- one of the keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry above reports -- so `from_data` dropped it with an `UnknownFieldWarning` and rebuilt every header with the host-default `d4c3b2a1` magic. `_make_data` now returns `byteorder` and `nanosecond`, and every byte order and timestamp resolution rebuilds unchanged. `Header._read_protos` also read four octets whatever `size` asked for ([#1125](https://github.com/JarryShaw/PyPCAPKit/issues/1125)). And `_make_magic` honoured `lilendian` and `bigendian` only as a pair, so `Header(bigendian=True)` wrote `d4c3b2a1` on a little-endian host; either flag alone now picks the byte order and overrides `byteorder`, and equal flags still raise `EndianError` ([#1144](https://github.com/JarryShaw/PyPCAPKit/issues/1144)).
+- the PCAP `Header` could not rebuild itself. `_make_magic`'s explicit-endian branch ignored `nanosecond` ([#1096](https://github.com/JarryShaw/PyPCAPKit/issues/1096)), and `_make_data` returned a `magic_number` that `make()` does not take -- one of the keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry above reports -- so `from_data` dropped it with an `UnknownFieldWarning` and rebuilt every header with the host-default `d4c3b2a1` magic. `_make_data` now returns `byteorder` and `nanosecond`, and every byte order and timestamp resolution rebuilds unchanged. `Header._read_protos` also read four octets whatever `size` asked for ([#1125](https://github.com/JarryShaw/PyPCAPKit/issues/1125)), and read them little-endian in a big-endian file; it now follows the file's byte order ([#2312](https://github.com/JarryShaw/PyPCAPKit/issues/2312)). And `_make_magic` honoured `lilendian` and `bigendian` only as a pair, so `Header(bigendian=True)` wrote `d4c3b2a1` on a little-endian host; either flag alone now picks the byte order and overrides `byteorder`, and equal flags still raise `EndianError` ([#1144](https://github.com/JarryShaw/PyPCAPKit/issues/1144)).
 - TCP's NS flag was written by `make()` but never parsed: it had been commented out of `Flags`, the reader and `_make_data`, with no reason recorded. Per the owner's ruling it is exposed rather than removed, so `info.flags.ns` exists and `from_data` keeps the bit ([#1098](https://github.com/JarryShaw/PyPCAPKit/issues/1098)).
 - `from_data` round trips. `Frame._make_data()` returned `ts_src` and `L2TPv2._make_data()` `prio`, the other keys the [#617](https://github.com/JarryShaw/PyPCAPKit/issues/617) entry reports, so `from_data` discarded both with an `UnknownFieldWarning` and an L2TPv2 priority bit was lost; they are now `ts_sec` and `priority` ([#1097](https://github.com/JarryShaw/PyPCAPKit/issues/1097)). And `ProtocolBase.from_data()` took no keywords, so `Frame.from_data` and `PCAPNG.from_data`, which need `num`/`header` and `num`/`sct`/`ctx`, could not be called at all. Extra keywords now reach the construction, overriding `_make_data`'s, under the construction-keyword check, so a typo raises `UnsupportedCall` ([#1126](https://github.com/JarryShaw/PyPCAPKit/issues/1126)). `HTTP.from_data` always raised `invalid HTTP version: 0`, looking up a `version` key neither versioned data class has; it now dispatches on the data's class. HTTP/2's `_make_data` also returned the payload `length`, which `ProtocolBase` takes as the parse length, so even a direct call failed as shorter than the 9-octet frame header; `make` now computes it, and every HTTP frame in the sample captures rebuilds byte for byte ([#1154](https://github.com/JarryShaw/PyPCAPKit/issues/1154)).
 - Quick-Start options did not round-trip. The IPv4, HOPOPT and IPv6-Opts readers took the function from `schema.func`, which only a parsed schema carries, so a built option raised `AttributeError`; HOPOPT and IPv6-Opts also sized the 8-octet option as 5, misreading the nonce with a `packet length < 0` warning ([#1091](https://github.com/JarryShaw/PyPCAPKit/issues/1091)). TCP's Quick-Start Response reader read one octet past the option, and its maker produced a negative rate code below 40 kbps; such a rate now rounds down to zero, which [RFC 4782](https://datatracker.ietf.org/doc/html/rfc4782) permits ([#1119](https://github.com/JarryShaw/PyPCAPKit/issues/1119)).

docs/source/changelog/1.5.0.rst

@@ -2397,7 +2397,9 @@ Fixed
   :exc:`.UnknownFieldWarning` and rebuilt every header with the host-default
   ``d4c3b2a1`` magic. ``_make_data`` now returns ``byteorder`` and ``nanosecond``, and
   every byte order and timestamp resolution rebuilds unchanged. ``Header._read_protos``
-  also read four octets whatever ``size`` asked for (:issue:`1125`). And ``_make_magic``
+  also read four octets whatever ``size`` asked for (:issue:`1125`), and read them
+  little-endian in a big-endian file; it now follows the file's byte order
+  (:issue:`2312`). And ``_make_magic``
   honoured ``lilendian`` and ``bigendian`` only as a pair, so ``Header(bigendian=True)``
   wrote ``d4c3b2a1`` on a little-endian host; either flag alone now picks the byte order
   and overrides ``byteorder``, and equal flags still raise :exc:`.EndianError`

pcapkit/protocols/misc/pcap/header.py

@@ -339,14 +339,18 @@ def _make_data(cls, data: 'Data_Header') -> 'dict[str, Any]':  # type: ignore[ov
     def _read_protos(self, size: int) -> 'Enum_LinkType':
         """Read next layer protocol type.
 
+        Reads ``size`` octets as an unsigned integer in the file's byte order
+        (:attr:`byteorder`), falling back to little-endian before :meth:`read`
+        has set it (:issue:`2312`).
+
         Arguments:
             size: buffer size
 
         Returns:
             Link layer protocol enumeration.
 
         """
-        _byte = self._read_unpack(size, lilendian=True)
+        _byte = self._read_unpack(size, lilendian=getattr(self, '_byte', 'little') == 'little')
         _prot = Enum_LinkType.get(_byte)
         return _prot
 

tests/protocols/misc/pcap/test_header_magic_protos_unit.py

@@ -56,6 +56,32 @@ def test_read_protos_reads_size_bytes(self) -> None:
                 self.assertEqual(header._read_protos(size), LinkType.ETHERNET)
                 self.assertEqual(header._file.tell(), size)
 
+    def test_read_protos_honours_file_byte_order(self) -> None:
+        # #2312: a big-endian file was read as little-endian.
+        from pcapkit.const.reg.linktype import LinkType
+        from pcapkit.protocols.misc.pcap.header import Header
+
+        for byteorder in ('big', 'little'):
+            for size in (1, 2, 4):
+                with self.subTest(byteorder=byteorder, size=size):
+                    header = object.__new__(Header)
+                    header._byte = byteorder
+                    header._file = io.BytesIO(int(LinkType.ETHERNET).to_bytes(size, byteorder) + b'\xff' * 4)
+                    self.assertEqual(header._read_protos(size), LinkType.ETHERNET)
+                    self.assertEqual(header._file.tell(), size)
+
+    def test_read_protos_parsed_big_endian_header(self) -> None:
+        # #2312: the byte order comes from the magic number ``read`` parsed.
+        from pcapkit.const.reg.linktype import LinkType
+        from pcapkit.protocols.misc.pcap.header import Header
+
+        raw = Header(bigendian=True, network=LinkType.ETHERNET).data
+        self.assertEqual(raw[:4], b'\xa1\xb2\xc3\xd4')
+        header = Header(io.BytesIO(raw))
+        self.assertEqual(header.byteorder, 'big')
+        header._file = io.BytesIO(b'\x00\x00\x00\x01')
+        self.assertEqual(header._read_protos(4), LinkType.ETHERNET)
+
 
 if __name__ == '__main__':
     unittest.main()