Skip to content

fix(protocols): from_data keeps declared lengths on truncated captures - #1166

Merged
JarryShaw merged 1 commit into
mainfrom
fix/1155-declared-lengths
Oct 7, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
fix/1155-declared-lengths

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner
  • Searched for similar pull requests
  • Followed the coding style (make pylint, make mypy, make isort) — not run
  • make test passes, and a test case covers the change — ran the new module plus test_ipv4_unit, test_ipv6_unit, test_tcp_udp_unit, test_udp_runtime, tests/project
  • Added a changelog entry under docs/source/changelog/ and regenerated CHANGELOG.md, if the change is user-visible — N/A, added centrally after the wave

What is the purpose of your pull request?

  • fix — corrects a defect

Description of your pull request and other information

Closes #1155, per the maintainer's ruling: on a truncated capture from_data keeps each length field as it was on the wire.

IPv4.make, UDP.make and IPv6.make computed the length from the payload handed to them, so a rebuild wrote the recorded length. Each now takes it explicitly — total_length for IPv4/UDP, payload_length for IPv6 — and _make_data passes the parsed one through. Omitting it computes as before, so IPv4(payload=...) is unchanged.

Public make() behaviour: additive only. A new optional keyword, appended before payload, with the old computation as its default; no existing call changes its output. Not a breaking change. _make_data gains a key, which is why three existing hand-built data stubs needed the field added (test_ipv4_unit, test_ipv6_unit, test_tcp_udp_unit — the UDP edit is two lines and does not touch the TCP options assertions).

Probe, X.from_data(x.info).data == x.data on the truncated frames:

frame before after
big_endian.pcap #3 IPv4 450004a2… → 45000052… matches
big_endian.pcap #3 UDP 048e → 003e matches
big_endian_nanosecond.pcap, little_endian.pcap same matches

Over all of examples/captures/, header-isolated: IPv4 mismatches 4 → 1, UDP 8 → 1, IPv6 0 → 0, with no new mismatches. The one remaining case is test.pcapng frame 3, cut inside the IPv4 header (18 of 20 octets), so UDP was parsed from zero bytes and its whole header is fabricated — a separate defect, not this one; the declared 300 is now preserved and the rebuilt prefix matches the 18 captured octets.

Reverting the fix: 11 failed, 4 passed, 39 subtests passed. With it: 6 passed, 48 subtests passed.

IPv4, IPv6 and UDP computed their length field from the payload `make()`
was handed, so a `from_data` rebuild of a capture cut short by the snaplen
wrote the recorded length instead of the declared one -- 1186 became 82 on
`big_endian.pcap` frame 3, and the UDP length 0x048e became 0x003e.

Each `make()` now takes the length explicitly (`total_length` for IPv4 and
UDP, `payload_length` for IPv6) and writes it as given; `_make_data` hands
over the parsed one. Omitting it computes the length as before, so a plain
`IPv4(payload=...)` is unchanged.

Closes #1155
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Oct 7, 2026
@JarryShaw JarryShaw added fix Pull requests that fix a defect (fix: subject prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Oct 7, 2026
@github-actions

github-actions Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Coverage: 88.87% (unit tier, Python 3.14, 41a67cc18, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 4 93.55%
pcapkit/const 18757 1035 2342 846 90.23%
pcapkit/corekit 1878 91 580 22 94.34%
pcapkit/dumpkit 142 0 42 0 100.00%
pcapkit/foundation 2422 104 842 40 94.24%
pcapkit/interface 112 7 40 5 92.11%
pcapkit/protocols 15710 188 3980 165 98.18%
pcapkit/toolkit 487 65 144 1 85.42%
pcapkit/utilities 429 4 122 4 98.55%
pcapkit/vendor 4409 2359 1006 158 42.84%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw

Copy link
Copy Markdown
Owner Author

Cross-review verdict on 41a67cc18: GOOD TO GO (ran on Sonnet; author Opus). Also labelled breaking, because the pack path changes.

  • Root cause confirmed: on main, IPv4.make writes ihl * 4 + len(payload) (ipv4.py:435) and UDP.make writes 8 + len(payload) (udp.py:200). A truncated capture therefore rebuilds with a shorter length: 04a2→0052 and 048e→003e on frame 3 of big_endian*.pcap and little_endian.pcap. Merged with main, all three rebuild byte-exact.
  • Sweep, with main merged in: header mismatches drop from 230 to 1 for IPv4, 8 to 1 for UDP, and 189 to 0 for IPv6, with zero new mismatches. The PR body's 4/8/0 understated the improvement. The one remaining case is test.pcapng frame 3; see below.
  • Default path unchanged: a 300-case fuzz of fresh make() builds with no length keyword is byte-identical to main.
  • No collisions or positional callers: every .make( in pcapkit/ goes through **kwargs, and the IPv4 local was cleanly renamed to options_length.
  • Why breaking: from_data no longer recomputes lengths, which is what the ruling asked for. The new parameter also sits before payload and is not keyword-only, so a positional UDP.make(..., payload) call would shift. Nothing in the repo calls it that way.
  • Fails without the fix: 11 tests fail. The edited stubs only add the new key.
  • Follow-up, not this PR: test.pcapng frame 3 is cut at 18 of 20 IPv4 header octets. The rebuild then fabricates a full 8-octet UDP header from an empty, all-zero parse. Filed as fix(protocols): a capture cut inside the IPv4 header rebuilds with a fabricated UDP header #1167.

@JarryShaw JarryShaw added breaking Breaks public-facing behaviour or API (apply alongside the type label) review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Oct 7, 2026
@JarryShaw
JarryShaw merged commit 99a9ced into main Oct 7, 2026
42 checks passed
@JarryShaw
JarryShaw deleted the fix/1155-declared-lengths branch October 7, 2026 01:46
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Oct 7, 2026
JarryShaw added a commit that referenced this pull request Oct 7, 2026
- 4 new 1.5.0 entries: HTTP/2 sample stream identifiers (#1161),
  TCP option re-padding (#1163, breaking), format='pcap' output
  (#1164), declared lengths on truncated captures (#1166, breaking)
- extend the PCAP Header (#1159), HIP (#1160) and from_data
  round-trip (#1162) entries rather than duplicating them
- regenerate CHANGELOG.md with util/changelog_md.py
- process.rst: entry-count pin re-measured, 189 -> 193

tests/project: conventions_doc_claims, changelog_md and
documentation_claims pass; changelog_md.py --check is in step.
JarryShaw added a commit that referenced this pull request Oct 7, 2026
)

`_make_data` returned the parsed Length under `length`, which
`ProtocolBase.__init__` consumes as the parse length, so it never reached
`make()` and the rebuild cleared `L` and omitted the Length octets.
`make()` now takes it as `total_length`, as #1166 did for IPv4 and UDP,
and `_make_data` passes it under that key.

`test_link_unit` called `make(length=20)` directly and is updated to the
new keyword. All 24 T/L/S/O(+pad) combinations now round-trip.

Closes #1172
@JarryShaw JarryShaw added this to the PyPCAPKit v1.5 milestone Oct 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) fix Pull requests that fix a defect (fix: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

design(protocols): should from_data keep declared lengths on truncated captures?

1 participant