Skip to content

perf(const): import the const enum packages lazily (#2729) - #2735

Merged
JarryShaw merged 1 commit into
mainfrom
perf/2729-lazy-const
Nov 3, 2026
Merged

JarryShaw merged 1 commit into
mainfrom
perf/2729-lazy-const

Conversation

@JarryShaw

Copy link
Copy Markdown
Owner

What is the purpose of your pull request?

  • perf — changes performance, not behaviour

This applies the owner's rulings: the pcapkit.const enum packages import lazily, the AppType registry loads SCTP and DCCP lazily the way it already loads TCP and UDP, and _Registries stops being a dict subclass (#2726).

  • pcapkit.utilities.lazy.lazy_exports is one shared helper. It gives a package a __getattr__ and __dir__ built from its export table. Every const package uses it. from pcapkit.const.x import Y, attribute access, dir(), __all__, star-imports, the AttributeError messages and member pickles all match main, and a cross-tree fingerprint shows no difference.
  • AppType: SCTP and DCCP are lazy.
  • _Registries is an @info_final Info. It stays lazy and becomes immutable. It is undocumented, and nothing in pcapkit mutated it.

Measured, plainly. import pcapkit is not measurably faster: 686 ms on main against 679 ms here, the median of 9 fresh processes. The protocol modules import most const modules at module level, so the saving waits on #2730. import pcapkit.const on its own, with the parent package stubbed out, drops from 189 ms to 52 ms. Treat this as groundwork for #2730 rather than a speed-up on its own.

Closes #2729

Every pcapkit.const package imported all of its enumeration modules with
itself, and pcapkit.const star-imported every package, so touching one
enumeration loaded them all. Only the TCP and UDP app-type registries
loaded on first use (#1538).

- utilities.lazy: lazy_exports builds a package's PEP 562 __getattr__ and
  __dir__ from an exports table, or from the __all__ of subpackages it
  re-exports; it generalises the #1538 hooks.
- const: each package names its exports under TYPE_CHECKING and binds the
  hooks, and pcapkit.const re-exports its subpackages through them, so
  importing it loads no enumeration module.
- const.reg.apptype: SCTP and DCCP load on first use as TCP and UDP do.
  _Registries is an immutable Info keyed by TransportProtocol rather than a
  dict subclass (#2726); AppType.__registries__ is typed as a Mapping, in
  the crawler template too.
- tests: conftest warms every const module; the #1538 tests cover SCTP,
  DCCP, the Info registries and a bare import of pcapkit.const; new sweeps
  hold every const package's __all__, dir() and member pickles to the eager
  behaviour; the #2656 test reads the lazy_exports call.
- docs: a page for pcapkit.utilities.lazy, and the ruling on
  const-vendor-pairing.

The new and changed laziness tests fail on 468f9c2 without the pcapkit/
change; the const, vendor, utilities, corekit, protocols root and schema,
and project modules pass, as do the const and utilities unittest legs.
@JarryShaw JarryShaw added this to the PyPCAPKit v1.6 milestone Nov 3, 2026
@JarryShaw JarryShaw added perf Pull requests that improve performance (perf: subject prefix) review: good-to-go Cross-review at the current head says ready; CI state is separate labels Nov 3, 2026
@JarryShaw

Copy link
Copy Markdown
Owner Author

GOOD TO GO at 562e5c19b, from a Sonnet review.

  • Behaviour is unchanged. A cross-tree fingerprint of every const package covers __all__, dir(), star-imports, hasattr, the AttributeError message, member pickles and submodule imports, and it matches main.
  • The laziness tests run in fresh subprocesses, so warming every module in conftest does not hide a regression.
  • Regenerating with the crawlers keeps the packages lazy.
  • Payoff: there is no measurable speed-up on import pcapkit until perf(protocols): import the protocol, schema and data packages lazily #2730, so this is groundwork. Two non-blocking notes:
    • __registries__ carries a type ignore, because Info is typed with str keys.
    • It overlaps slightly with ModuleDescriptor, which utilities cannot import.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Coverage: 94.44% (unit tier, Python 3.15, 562e5c19b, Unit Tests run success)

Package Statements Missed Branches Partial Cover
pcapkit (top level) 104 4 20 2 95.16%
pcapkit/const 20918 175 2908 156 98.57%
pcapkit/corekit 2861 76 916 37 96.80%
pcapkit/dumpkit 282 3 100 3 98.43%
pcapkit/foundation 3496 26 1214 32 98.77%
pcapkit/interface 138 4 52 3 96.32%
pcapkit/protocols 22613 150 5862 123 99.03%
pcapkit/toolkit 961 26 342 8 96.62%
pcapkit/utilities 559 7 150 5 98.31%
pcapkit/vendor 5590 2286 1280 214 55.75%

Per-file detail: the coverage-html artifact of this run.

@JarryShaw
JarryShaw merged commit 3776067 into main Nov 3, 2026
65 checks passed
@github-project-automation github-project-automation Bot moved this from In review to Done in PyPCAPKit Nov 3, 2026
@JarryShaw
JarryShaw deleted the perf/2729-lazy-const branch November 3, 2026 16:27
@JarryShaw JarryShaw removed the review: good-to-go Cross-review at the current head says ready; CI state is separate label Nov 3, 2026
JarryShaw added a commit that referenced this pull request Nov 3, 2026


- Breaking: a short CALIPSO, SMF_DPD or MPL Opt Data Len raises ProtocolError,
  where 1.5.x raised FieldValueError, and a cut after the type octet reads
  short (#2737), a new entry; AppType.__registries__ is an Info, into the
  #2734 entry (#2735).
- Changed: const packages import their enumerations lazily through
  pcapkit.utilities.lazy.lazy_exports (#2735); the docs extra pins
  sphinx-autodoc-typehints 3.17.4 (#2742), as 1.5.0 recorded its pin.
- Fixed: the dumps keep a field renamed for clashing with an Info builtin
  name (#2736).
- Folded: the GraalPy leg's timeout and shards into the CI-legs entry
  (#2741); the #2735 and #2737 rulings into the conventions entry.
- No entry for the docs-only #2744 and #2746.
- 1.6.0's :func:`bytes` role is :class:`bytes`, so the changelog pages build
  with no nitpicky warning. Old values measured against v1.5.1.
- CHANGELOG.md regenerated; util/changelog_md.py --check passes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

perf Pull requests that improve performance (perf: subject prefix)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

perf(const): import the const enum packages lazily

1 participant