|
| 1 | +# Demonstration Scripts |
| 2 | + |
| 3 | +A tour of `pcapkit` by example: each script here does one thing and prints or dumps |
| 4 | +the result, so you can get a thorough view of the library either on how to use it or |
| 5 | +on what it can do. They are demos rather than tests -- `pytest` does not collect |
| 6 | +this directory (`testpaths = ["tests"]`) and nothing in CI runs them, so they are |
| 7 | +also where API drift goes unnoticed the longest. Run them after changing anything |
| 8 | +public. |
| 9 | + |
| 10 | +## Running them |
| 11 | + |
| 12 | +Every script reads its capture with a path relative to its own directory, so **run |
| 13 | +them from inside** `examples/legacy_smoke/`, not from the repository root: |
| 14 | + |
| 15 | +```shell |
| 16 | +cd examples/legacy_smoke |
| 17 | +python test_basic.py |
| 18 | +``` |
| 19 | + |
| 20 | +The captures live in `../captures/`. Only `in.pcap` and `dhcp.pcapng` are |
| 21 | +committed; the rest are generated, so build them once before running anything else: |
| 22 | + |
| 23 | +```shell |
| 24 | +python ../generators/make_samples.py # or: make samples, from the repository root |
| 25 | +``` |
| 26 | + |
| 27 | +## The scripts |
| 28 | + |
| 29 | +| Script | What it demonstrates | |
| 30 | +| --- | --- | |
| 31 | +| [test_basic](test_basic.py) | the smallest useful call -- extract `in.pcap` to a tree-view text file | |
| 32 | +| [test_extractor](test_extractor.py) | `pcapkit.extract` dumping the same capture as plist, JSON and tree | |
| 33 | +| [test_file](test_file.py) | reading from an already-open binary file object rather than a path | |
| 34 | +| [test_api](test_api.py) | most of the keyword surface at once: per-frame files, reassembly and flow tracing | |
| 35 | +| [test_tcp](test_tcp.py) | extraction of TCP packets | |
| 36 | +| [test_ipv6](test_ipv6.py) | extraction of IPv6 packets, one output file per frame | |
| 37 | +| [test_pcapng](test_pcapng.py) | extraction of a PCAP-NG capture, which uses a different engine | |
| 38 | +| [test_http](test_http.py) | iterating frames manually and testing `pcapkit.HTTP in frame` | |
| 39 | +| [test_reassembly](test_reassembly.py) | TCP payload reassembly, writing the reassembled datagrams out | |
| 40 | +| [test_ip_reasm](test_ip_reasm.py) | IPv4 fragment reassembly | |
| 41 | +| [test_ipv6_reasm](test_ipv6_reasm.py) | IPv6 fragment reassembly | |
| 42 | +| [test_analyse](test_analyse.py) | analysis of the application layer *after* reassembly, printing the recovered HTTP messages | |
| 43 | +| [test_trace](test_trace.py) | tracing TCP flows and dumping the flow index | |
| 44 | +| [test_engine](test_engine.py) | the same capture through each extraction engine, side by side | |
| 45 | +| [test_time](test_time.py) | timing each engine, reported as milliseconds per packet | |
| 46 | +| [test_profile](test_profile.py) | cProfile stats for one extraction; `make profile` renders the call graph | |
| 47 | +| [test_perf](test_perf.py) | the same comparison under the `pyperf` benchmark harness | |
| 48 | +| [test_stream](test_stream.py) | extracting a live capture streamed from `tcpdump` on stdin | |
| 49 | +| [test_stream_askpass](test_stream_askpass.py) | the same, with `sudo -A` for a non-interactive password | |
| 50 | + |
| 51 | +`_engine_support.py` is not a demo -- it is the shared helper the engine demos use |
| 52 | +to work out whether an engine can run on this host. |
| 53 | + |
| 54 | +## What needs more than pcapkit |
| 55 | + |
| 56 | +Most of these run with nothing but `pcapkit`. These do not: |
| 57 | + |
| 58 | +- **test_engine, test_time, test_perf** exercise the `dpkt`, `scapy` and |
| 59 | + `pyshark` engines. Those are optional packages, and `pyshark` additionally |
| 60 | + drives Wireshark's `tshark` binary. An engine that is missing is reported as |
| 61 | + skipped, with the reason, and the rest still run -- see `_engine_support.py`. Two |
| 62 | + things worth knowing: |
| 63 | + |
| 64 | + - `pyshark` 0.6 does not work on Python 3.14 at all. It asks |
| 65 | + `asyncio.get_event_loop_policy().get_event_loop()` for a loop on a thread that |
| 66 | + has none, which 3.14 no longer creates implicitly, so every extraction through it |
| 67 | + raises `RuntimeError: There is no current event loop in thread 'MainThread'` |
| 68 | + before `tshark` is even reached. That is a pyshark bug, not a pcapkit one, and |
| 69 | + there is nothing to configure around it. |
| 70 | + - When an engine's *package* is missing, `pcapkit.extract` does not fail. It warns |
| 71 | + and falls back to pcapkit's own parser, so the extraction succeeds under the wrong |
| 72 | + engine. The demos name the driver that actually ran, so a fallback is visible |
| 73 | + rather than being reported as a healthy result. |
| 74 | + |
| 75 | +- **test_perf** needs the `pyperf` harness, which `pcapkit` does not depend on: |
| 76 | + `python -m pip install pyperf`. Without it the script says so and stops. |
| 77 | + `test_time.py` does the same comparison with no extra dependency. |
| 78 | +- **test_profile**'s `make profile` target needs `gprof2dot`, `graphviz` and |
| 79 | + `snakeviz` to render the call graph. The script itself only needs `pcapkit`. |
| 80 | +- **test_stream, test_stream_askpass** run `sudo tcpdump` against a live interface, |
| 81 | + so they need root and cannot run unattended. `INTERFACE` at the top of each is |
| 82 | + `en0` (macOS); on Linux it is usually `eth0` or `wlan0`. They buffer the live |
| 83 | + capture into a temporary file -- never into `../captures/`, which holds generated |
| 84 | + captures the test suite pins. |
| 85 | + |
| 86 | +## Regenerating the reference reports |
| 87 | + |
| 88 | +Four files in `../captures/` are reports rather than inputs -- `out.json`, |
| 89 | +`out.plist`, `out.txt` and `pcapng.txt` -- and they are produced by two of the |
| 90 | +scripts here. **Git does not track them**, so a fresh clone does not have them. Build |
| 91 | +them the way you would any other generated capture, and again after a change to how |
| 92 | +`pcapkit` renders one: |
| 93 | + |
| 94 | +```shell |
| 95 | +cd examples/legacy_smoke && make fixtures |
| 96 | +``` |
| 97 | + |
| 98 | +That is the whole recipe. It runs, equivalently: |
| 99 | + |
| 100 | +```shell |
| 101 | +TZ=UTC python test_extractor.py # ../captures/out.json, out.plist, out.txt <- in.pcap |
| 102 | +TZ=UTC python test_pcapng.py # ../captures/pcapng.txt <- dhcp.pcapng |
| 103 | +``` |
| 104 | + |
| 105 | +Both *inputs* are committed, so this works on a fresh clone with no `make samples` |
| 106 | +first. Use `make fixtures PYTHON=../../.venv/bin/python` to pick a specific |
| 107 | +interpreter. |
| 108 | + |
| 109 | +These four were tracked once, and #685 is why they are not any more. Nothing reads a |
| 110 | +report back -- no test asserts on one, and the docs only quote them as prose -- so a |
| 111 | +parser change that alters how a frame renders leaves the committed copy quietly wrong, |
| 112 | +with no failure anywhere to say so. `pcapng.txt` sat recording `packet -> NIL` for |
| 113 | +four blocks that had started carrying their captured octets, and the staleness was |
| 114 | +found by reading the file rather than by anything breaking. Generating them on demand |
| 115 | +means the copy on your disk describes the tree you are standing in, which was the only |
| 116 | +thing that made them worth looking at. |
| 117 | + |
| 118 | +`TZ=UTC` **is not decoration.** `pcapkit` renders frame timestamps in the host's |
| 119 | +local zone, so an unpinned run rewrites every timestamp line in all four files with |
| 120 | +wherever it happened to be run -- which is how these reports came to disagree with |
| 121 | +each other while they were still tracked, the PCAP three having been generated at |
| 122 | +UTC-05:00 and `pcapng.txt` at UTC+08:00. Pinning UTC makes the output reproducible on |
| 123 | +any host, so two runs can be diffed against each other. It also happens to be the only |
| 124 | +zone in which PCAP-NG's `timestamp_epoch` is the true UNIX epoch: |
| 125 | +`PCAPNG._read_timestamp` adds the zone's UTC offset to the value it returns, so a |
| 126 | +report generated anywhere else bakes in that offset. |
0 commit comments