Skip to content

refactor(protocols)!: refuse fixed-size octets and timestamp counts the makers masked (#2676) - #2691

Merged
JarryShaw merged 1 commit into
mainfrom
refactor/2676-maker-masks
Nov 3, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
refactor/2676-maker-masks

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • refactor — changes neither behaviour nor performance

Description of your pull request and other information

The rest of #2676, after the registry-code bullet, which #2594's rework takes.

  • Fixed-size octets: a caller's octets for a fixed-size field go through _make_octets and raise ProtocolError at any other width, instead of being padded or cut. This covers the checksums of IPv4, MH, HIP, ICMP, IGMP, IPX and OSPF, the CALIPSO checksum, MH cookies and keygen tokens, HIP opaque and payload data, PCAP-NG ZigBee keys, IPX addresses, and the reserved and unused octets of the IPv6 extensions, MH, HIP, ICMP, IGMPv3, OSPF and IP_Pseudo. A source sweep in the tests finds every such field a maker writes, so a new one fails without a row.
  • IPv4 timestamp option: a slot count past what the option holds raises, instead of being clamped. The default is derived from the form, so both defaults build the same octets as before.
  • Parsed models rebuild as captured: a parsed MH message, IPv6-Route type, IPv6-Frag header or OSPF authentication model keeps its octets as captured. Only a caller's value is checked, so a capture cut inside them still rebuilds.
  • Docs: protocol.rst documents the shared value helpers.
  • MH ipv4_dhcp.mode=True is unchanged: it is a one-bit field, which takes a bool.

Breaking: released makers that padded or cut a fixed-size value now raise.

Closes #2676

@JarryShaw JarryShaw added this to the PyPCAPKit v1.6 milestone Nov 2, 2026
@JarryShaw JarryShaw added breaking Breaks public-facing behaviour or API (apply alongside the type label) refactor Restructuring for its own sake — neither a fix nor a new capability (refactor: prefix) review: running A cross-review is in flight against the current head - no verdict yet labels Nov 2, 2026
@JarryShaw JarryShaw moved this to In review in PyPCAPKit Nov 2, 2026
@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Nov 2, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 9ee57bdf8, from a Sonnet cross-review. Caller values, defaults, the timestamp counts and the parse output are all confirmed, and captures are identical to main. What has to change:

  1. The rebuild exemption lets a caller-built model through. It treats any data model as a rebuild: MH msg_reserved, IPv6-Route type 0 and type 2, and OSPF crypto auth. So a hand-built model with a wrong-width reserved field is padded or cut instead of refused. Follow IPv6_Frag.from_data: give the maker a full-width value, let replay_short_read trim, and check every value. At the very least, key the exemption on __short_read__.
  2. No test pins the exemptions. Removing any of them fails nothing. Add direct-parse cut tests for each exempted site, and a test that a wrong-width model is refused.
  3. CI: gate ast.unparse (test(project): the #1878 engine-stub tests call ast.unparse, which Python 3.8 lacks #1899), and add the new test to project-tests in unit-tests.yml.

@JarryShaw
JarryShaw force-pushed the refactor/2676-maker-masks branch from 9ee57bd to 6ca69b2 Compare November 3, 2026 00:28
@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Nov 3, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

Pushed the fix round 9ee57bdf8→6ca69b2ec, rebased onto 57a6c366d.

  • Every value is now checked: _make_octets zero-fills a short value only when the model carries the built-in __short_read__ record. A caller-built model is checked like any other value.
  • Each exempted site is pinned: direct-parse cut tests cover MH, IPv6-Route 0 and 2, IPv6-Frag and OSPF, and wrong-width models are refused.
  • CI: ast.unparse is replaced, and the docs check moved under tests/project.

A Sonnet re-review is running.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Nov 3, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at 6ca69b2ec, from a Sonnet re-review. Most of it is confirmed:

  • every cut capture at every offset rebuilds exactly;
  • the CI fixes and rebase resolutions are right;
  • captures are identical;
  • 284 modules pass.

One hole remains: the zero-fill fires whenever the model carries any __short_read__, not only one in the reserved field. So editing msg_reserved (MH) or reserved (IPv6-Route 0) to a short value, on a model cut somewhere else, is padded instead of refused.

  1. Tie the fill to the record naming this field, or to a cut that falls at or inside it.
  2. Add Frag to the wrong-width test. An unconditional Frag fill survives today.
  3. Add the "record elsewhere, reserved edited short" case for MH and Route.

@JarryShaw
JarryShaw force-pushed the refactor/2676-maker-masks branch from 6ca69b2 to ea9935d Compare November 3, 2026 02:08
@JarryShaw

Copy link
Copy Markdown
Owner Author

Pushed the fix round 6ca69b2ec→ea9935d45, still one commit on 57a6c366d.

  • The fill is now Protocol._fill_short_read. It zero-fills a short value only when this model's own __short_read__ record shows the cut ended inside the field or before it. A value cut elsewhere, or of any other width, still goes to _make_octets and is refused. Both of the re-review's repros now raise ProtocolError.
  • This covers the 8 MH msg_reserved messages, IPv6-Route types 0 and 2, IPv6_Frag and OSPF. _make_msg_brr needs no fill, because its reserved field is an integer.
  • New cases cut past each field and shorten it. Run against 6ca69b2ec, they fail 12 subtests.

A Sonnet re-review is running; review: running.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Nov 3, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

NEEDS CHANGES at ea9935d45, from a Sonnet re-review that I reproduced. The fix for the previous finding holds: _fill_short_read fills only on this model's own record, and both earlier repros are refused.

  • New regression: an OSPF cut at 16 or 17, inside the cryptographic-authentication reserved octets, rebuilds from info but raises ProtocolError ("must be 2 octets") from info.to_dict(). OSPF._make_data fills only when auth is a Data_CryptographicAuthentication, and to_dict() turns it into a mapping. Both cuts rebuild on main.
  • Fix: apply the fill when auth is a mapping too, and extend test_ospf_cut_inside_authentication_reserved_octets to the to_dict form.
  • The full regression legs ran out of time in the review, so they still need a complete run.

@JarryShaw JarryShaw added review: needs-changes Cross-review at the current head says changes are required; see the verdict comment and removed review: running A cross-review is in flight against the current head - no verdict yet labels Nov 3, 2026
…he makers masked (#2676)

- _make_octets: a caller's octets for a fixed-size field raise ProtocolError
  at any other width instead of being padded or cut -- the IPv4, MH, HIP,
  ICMP, IGMP, IPX and OSPF checksums, the CALIPSO checksum, MH cookies and
  keygen tokens, HIP opaque and payload data, PCAP-NG ZigBee keys, IPX
  addresses, and the reserved, unused and zero octets of HOPOPT, IPv6-Opts,
  IPv6-Frag, IPv6-Route, MH, HIP, ICMP, IGMPv3, OSPF and IP_Pseudo.
- _fill_short_read: a model's reserved octets are zero-filled before the
  check only where its short read ends at or before them, for the rebuild
  to cut back; any other model's are checked as a caller's are.
- IPv4 TS: a slot count past nine (list) or four (dict) raises instead of
  being cut; the default is derived from the form.
- Empty defaults of fixed-size fields become zero octets of their width.
- protocol.rst documents the shared value helpers and how makers use them.
- MH ipv4_dhcp.mode=True is unchanged: S is one bit, which takes a bool.
- Tests: a source sweep finds every fixed-size octet field a maker writes;
  every refusal fails on main; cut captures rebuild as captured; the widest
  values build and round-trip.
@JarryShaw
JarryShaw force-pushed the refactor/2676-maker-masks branch from ea9935d to d57c0a4 Compare November 3, 2026 04:06
@JarryShaw

Copy link
Copy Markdown
Owner Author

Pushed ea9935d45→d57c0a45f, rebased onto 596dcfaf8, still one commit.

A Sonnet re-review is running; review: running.

@JarryShaw JarryShaw added review: running A cross-review is in flight against the current head - no verdict yet and removed review: needs-changes Cross-review at the current head says changes are required; see the verdict comment labels Nov 3, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at d57c0a45f, from a Sonnet re-review.

@JarryShaw JarryShaw added review: good-to-go Cross-review at the current head says ready; CI state is separate and removed review: running A cross-review is in flight against the current head - no verdict yet labels Nov 3, 2026
@JarryShaw
JarryShaw merged commit 7fe9e8b into main Nov 3, 2026
63 checks passed
@JarryShaw
JarryShaw deleted the refactor/2676-maker-masks branch November 3, 2026 04:33
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Nov 3, 2026
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Nov 3, 2026
JarryShaw added a commit that referenced this pull request Nov 3, 2026
…s.pickle, FRAD's pad and the TCP and HIP maker fixes into 1.6.0 (#2449)

- Breaking: a fixed-size octet field given any other width raises, where
  1.5.x padded or cut it, and IPv4's Timestamp option refuses a slot count
  past what it holds, where 1.5.x cut it (#2691); folded into the IPv4 to HIP
  makers' range-check entry, with the ZigBee keys into the PCAP-NG entry. Old
  values measured against v1.5.1.
- Added: pcapkit.utilities.pickle, whose ReduceAsProtocol2 lets Info, Schema
  and the reassembly engines pickle under protocols 0 and 1 on GraalPy, and
  numbers._reduce_unregistered_member's removal (#2722).
- Fixed: TCP's raw-octet options, HIP's default version, HIP's ENCRYPTED IV
  and its cipher on parse (#2724).
- Folded into unreleased entries: FRAD's pad field and make(pad=) (#2721).
- No entry for the docstring-only 596dcfa, the test-only 149cced, or
  #2724's MH annotation.
- CHANGELOG.md regenerated; util/changelog_md.py --check passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking Breaks public-facing behaviour or API (apply alongside the type label) refactor Restructuring for its own sake — neither a fix nor a new capability (refactor: prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

refactor(protocols)!: range-check the registry codes, timestamp counts and byte fields the makers still mask

1 participant