|
1 | 1 | # CLAUDE.md |
2 | 2 |
|
3 | | -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 3 | +Guidance for future Claude Code sessions working in this repository. |
4 | 4 |
|
5 | | -## Phase 3b status (2026) |
| 5 | +## What this is |
6 | 6 |
|
7 | | -Neural differential cryptanalysis pipeline in `src/keeloq/neural/`, following |
8 | | -Gohr 2019 adapted for KeeLoq's 1-bit-per-round key schedule. CLI: |
| 7 | +2026 modernization of a 2015 KeeLoq cryptanalysis project. Two pipelines live in `src/keeloq/`: |
9 | 8 |
|
10 | | -- `keeloq neural train --rounds N --delta 0xΔ --samples M --out <ckpt>` |
11 | | -- `keeloq neural evaluate --checkpoint <ckpt> --rounds N` |
12 | | -- `keeloq neural recover-key --checkpoint <ckpt> --rounds N --diff-pair <c0>:<c1> --sat-pair <pt>:<ct>` |
13 | | -- `keeloq neural auto --rounds N --trained-depth D --samples M --checkpoint-out <path>` |
| 9 | +- **Algebraic / SAT** (Phase 1) — full-key recovery at 64 rounds / 4 pairs / 0 hints in ~0.25 s on an RTX 5090 via XOR-aware ANF encoding + CryptoMiniSat. The 2015 baseline (160 rounds / 25 hints / 2 pairs) took 14 hours; Phase 1's matrix benchmarks both for comparison. |
| 10 | +- **Neural differential** (Phase 3b) — Gohr-style ResNet-1D-CNN distinguisher + Bayesian 1-bit-per-round key recovery + SAT suffix. Uses the Phase 1 GPU bit-sliced cipher for training data (~10⁶ pairs/sec on a 5090). Distinguishers are not committed to the repo (training is local, reproducible via `keeloq neural auto`). |
14 | 11 |
|
15 | | -**Gohr-pattern constraint.** A distinguisher trained at depth D gives strong signal only on |
16 | | -D-round pairs. Peeling K rounds with a single distinguisher works well only for small K. For |
17 | | -deep-round attacks, train a family of distinguishers at strategic depths (e.g. D=56 for a |
18 | | -64-round attack peeling K=8 rounds). The `auto` subcommand handles this automatically. |
19 | | -`neural-target-bits` controls how many prefix key-bits the neural phase covers before handing |
20 | | -off to SAT. |
| 12 | +The original 2015 Python 2 scripts live untouched in `legacy/` and are exercised via Docker (`python:2.7`) parity tests. **Never modify files in `legacy/`.** |
21 | 13 |
|
22 | | -**Checkpoint policy.** Checkpoints are NOT committed to the repo by default (external GPU |
23 | | -contention prevented full training runs for d64.pt/d96.pt/d128.pt). Produce them via: |
| 14 | +Authoritative references: |
24 | 15 |
|
25 | | - uv run keeloq neural auto --rounds 64 --trained-depth 56 \ |
26 | | - --samples 10000000 --pairs 512 --checkpoint-out checkpoints/d64.pt |
27 | | - |
28 | | -The regression test `tests/test_neural_e2e_64r.py` auto-skips when `checkpoints/d64.pt` is |
29 | | -absent. Benchmark runner (`benchmarks/bench_neural.py` + `benchmarks/neural_matrix.toml`) is |
30 | | -smoke-safe against missing checkpoints. |
31 | | - |
32 | | -**Hybrid-attack CLI.** The `recover-key` subcommand takes two distinct argument streams: |
33 | | -`--diff-pair <c0>:<c1>` (differential ciphertext pairs for the neural distinguisher) and |
34 | | -`--sat-pair <pt>:<ct>` (plaintext:ciphertext pairs for the SAT suffix). These are separate |
35 | | -because a differential attack doesn't require known plaintexts; only the SAT phase does. |
36 | | - |
37 | | -**CUDA XOR limitation.** `rshift_cuda` for uint32 is not implemented in PyTorch 2.11 + CUDA |
38 | | -13. All XOR / delta application is done CPU-side before tensors are moved to the GPU. |
39 | | - |
40 | | -Checkpoints in `checkpoints/` with training metadata embedded. Results under |
41 | | -`docs/phase3b-results/`. Spec/plan in `docs/superpowers/`. |
42 | | - |
43 | | -## Phase 1 status (2026) |
| 16 | +- `docs/superpowers/specs/` — per-phase design docs (2026-04-22-phase1-foundation-design.md, 2026-04-22-phase3b-neural-cryptanalysis-design.md). |
| 17 | +- `docs/superpowers/plans/` — task-by-task implementation plans for each phase. |
| 18 | +- `README.md` — user-facing quickstart and project layout. |
44 | 19 |
|
45 | | -The repo is mid-migration from the 2015 Python 2 scripts (now in `legacy/`, frozen) to a |
46 | | -Python 3 modernized pipeline in `src/keeloq/`. Driver is `keeloq` (a Typer CLI, |
47 | | -installed via `uv sync --all-extras`). Key entry points: |
| 20 | +## Tooling and conventions |
48 | 21 |
|
49 | | -- `keeloq encrypt / decrypt` — the cipher, rounds-parameterized. |
50 | | -- `keeloq generate-anf | encode | solve | verify` — pipeline stages composable via Unix pipes (JSON between stages). |
51 | | -- `keeloq attack` — the pipeline in-process. Use `--pair pt:ct` (repeatable) for multi-pair attacks. |
52 | | -- `keeloq benchmark` — matrix runner driven by `benchmarks/matrix.toml`. |
| 22 | +- **Python 3.12**, packaged via `uv`. `uv sync --all-extras` to install everything including dev tools. |
| 23 | +- **Tests**: `pytest` with markers — `@pytest.mark.gpu` (requires CUDA), `@pytest.mark.slow` (multi-second end-to-end), `@pytest.mark.legacy` (requires Docker + `python:2.7`). Fast suite = `uv run pytest -m "not slow"` (~30 s on the 5090 box). |
| 24 | +- **Lint + types**: `ruff check` + `ruff format --check` + `mypy` (scoped to `src/keeloq`). Pre-commit convention: all three clean before every commit. |
| 25 | +- **TDD discipline**: tests first, confirm fail, implement, confirm pass, commit. Commit prefixes `test:`, `impl:`, `refactor:`, `feat:`, `docs:`, `ci:`, `chore:` — do NOT squash them. The plan docs list each task's expected red→green→commit cycle. |
53 | 26 |
|
54 | | -Strict TDD discipline. Commits are prefixed `test:`, `impl:`, `refactor:`, `feat:`, `docs:`, `ci:`, `chore:` — do NOT squash them. |
| 27 | +## Domain knowledge that's not obvious from the code |
55 | 28 |
|
56 | | -**Cross-validation TDD invariant.** Two encoders (`encoders/cnf.py`, `encoders/xor_aware.py`) |
57 | | -must recover the same key on the same inputs; `tests/test_encoders_agree.py` enforces this. |
58 | | -Variable-indexing bugs are the biggest risk in this domain, and this cross-check is the |
59 | | -primary defense. |
| 29 | +**KeeLoq bit convention is MSB-first.** `keeloq.cipher._state_bit(s, 0)` returns the MSB of a 32-bit state; `_key_bit(k, 0)` returns the MSB of a 64-bit key. This matches the 2015 Python 2 scripts' `list(PLAINTEXT)[0]` indexing. Cross-validation lives in `tests/test_compat.py` (parity vs. legacy scripts inside Docker). |
60 | 30 |
|
61 | | -**Legacy is frozen and runs via Docker.** Never modify files in `legacy/`. The compat tests |
62 | | -run the 2015 Python 2 scripts inside an ephemeral `python:2.7` container (not host |
63 | | -python2, which is EOL). `tests/compat_helpers.py` handles the docker invocation. Tests |
64 | | -mark `@pytest.mark.legacy`; they auto-skip if docker or the python:2.7 image is absent. |
| 31 | +**Three variable families in the ANF system** (shared with the 2015 scripts): |
65 | 32 |
|
66 | | -**Key-schedule quirk.** KeeLoq's key cycles at 64 rounds. Attacks at `rounds < 64` |
67 | | -fundamentally cannot constrain `K{rounds}..K63` — those bits must be hinted, or the |
68 | | -attack must run at rounds ≥ 64 with enough plaintext/ciphertext pairs to over-determine |
69 | | -the system. A clean 0-hint key recovery needs 64 rounds + ~4 pairs. This is why the |
70 | | -benchmark `matrix.toml` rows with rounds < 64 always pin `hint_bits >= (64 - rounds)`. |
| 33 | +- `K0..K63` — 64 key bits, shared across plaintext/ciphertext pairs. |
| 34 | +- `L{j}_p{i}` — the NLFSR state bit `j` for pair `i`. `L0..L31` is plaintext, `L32..L{rounds+31}` are intermediate state bits produced round by round. |
| 35 | +- `A{i}_p{j}`, `B{i}_p{j}` — linearization helpers that keep each round equation at degree ≤ 2 for the SAT encoder. `A_i = L_{i+31} · L_{i+26}`, `B_i = L_{i+31} · L_{i+1}`. Dropping or renaming them breaks the encoder contract. |
71 | 36 |
|
72 | | -**GPU bit-sliced cipher.** `src/keeloq/gpu_cipher.py` is a PyTorch bit-sliced KeeLoq |
73 | | -used as a property-test oracle. Requires CUDA; tests auto-skip on CUDA-less machines. |
74 | | -Works internally on int64 tensors because `rshift_cuda` isn't implemented for uint32 on |
75 | | -PyTorch 2.11 + CUDA 13; the public API uses uint32. |
| 37 | +The core nonlinear function is shared across cipher, ANF generator, and GPU cipher: |
76 | 38 |
|
77 | | -## What this repo is |
| 39 | + core(a,b,c,d,e) = d + e + ac + ae + bc + be + cd + de + ade + ace + abd + abc (mod 2) |
78 | 40 |
|
79 | | -Research code for an algebraic / SAT-based attack on a reduced-round (160-round) version of the KeeLoq block cipher, originally authored 2015. It is not a library or a product — it's a small pipeline of one-shot scripts that cooperate via files (`anf.txt`, `vars.txt`, the CNF output, `out.result`). |
| 41 | +**Cross-validation TDD invariant.** Two encoders (`encoders/cnf.py`, `encoders/xor_aware.py`) must recover the same key on the same inputs; `tests/test_encoders_agree.py` enforces this. Variable-indexing bugs are the biggest risk in this domain, and this independent-oracles cross-check is the primary defense. |
80 | 42 |
|
81 | | -The full cipher is 528 rounds; the 160-round variant is the target of the attack, and the scripts that generate equations (`sage-equations.py`, `sage-CNF-convert.txt`) are hard-coded for 160 rounds. |
| 43 | +**Cyclic key schedule at 64 rounds.** KeeLoq's key cycles every 64 rounds. At `rounds < 64`, bits `K_{rounds}..K_{63}` are never referenced and can't be recovered without being hinted. The `benchmarks/matrix.toml` rows with rounds < 64 always pin `hint_bits >= (64 - rounds)`. For neural attacks, `keeloq neural auto` auto-populates `extra_key_hints` for the unconstrained range. |
82 | 44 |
|
83 | | -## Python version |
| 45 | +**Decryption key index differs between the legacy 160- and 528-round scripts.** `legacy/keeloq-python.py` (528 rounds) uses `k[15]`; `legacy/keeloq160-python.py` (160 rounds) uses `k[31]`. This reflects the residual key offset after each round count (528 = 8·64 + 16 vs. 160 = 2·64 + 32). The modern `src/keeloq/cipher.py::decrypt` derives from the algebraic inverse of the round function, so a single parameterized implementation handles both — no need to replicate the k[15]/k[31] hack. |
84 | 46 |
|
85 | | -All `.py` files are **Python 2** (they use statement-form `print "..."`). Do not "modernize" syntax without an explicit request — running under Python 3 will SyntaxError. If you run scripts, use `python2`. |
| 47 | +**CUDA uint32 limitation.** PyTorch 2.11 + CUDA 13 does not implement `rshift_cuda` or `bitwise_xor` for `uint32` tensors. All such operations happen on int64 lanes internally (e.g., `src/keeloq/gpu_cipher.py`), or on CPU before casting to CUDA (training-data XOR, differential pair construction). Public APIs still use `uint32` for bit-pattern clarity. |
86 | 48 |
|
87 | | -## Attack pipeline (read this before editing) |
| 49 | +## Phase 3b specifics |
88 | 50 |
|
89 | | -The files form a sequential pipeline. Each step consumes the output of the previous one: |
| 51 | +**Gohr-pattern constraint.** A distinguisher trained at depth **D** gives strong signal only on pairs at depth **D**. Peeling **K** rounds with a single distinguisher works well only for small **K** (because intermediate-depth pairs fall outside the distinguisher's training distribution and signal degrades). For deep-round attacks, train a family of distinguishers at strategic depths (e.g. D=56 for a 64-round attack peeling K=8 rounds). The `auto` subcommand uses `--trained-depth` to make this explicit. |
90 | 52 |
|
91 | | -1. `keeloq160-python.py` — generates a known (plaintext, ciphertext) pair under a chosen key. The plaintext/key/ciphertext triple is then hand-copied into the next stage. |
92 | | -2. `sage-equations.py` — writes `anf.txt`: an ANF (Algebraic Normal Form) polynomial system over GF(2) encoding the round function, the known plaintext/ciphertext bits, and (optionally) key-bit hints. Emits one round-equation triple `(eq1, eq2, eq3)` per round for 160 rounds. |
93 | | -3. `polynomial-vars.py` — writes `vars.txt`: the variable list to paste into SageMath's `BooleanPolynomialRing()` declaration. |
94 | | -4. `sage-CNF-convert.txt` — a SageMath script (not Python 2; paste into `sage`) that uses `sage.sat.converters.polybori.CNFEncoder` + `DIMACS` to convert the ANF system to DIMACS CNF. |
95 | | -5. External: run `minisat main160.cnf out.result`. |
96 | | -6. `parse-miniSAT.py` — reads `out.result`, takes the first 64 literals (the key variables `K0..K63`), interprets `-` as 0, and prints the recovered key against the original. |
| 53 | +**Two distinct pair streams in the hybrid attack.** `hybrid_attack()` takes: |
97 | 54 |
|
98 | | -`keeloq-python.py` is the full 528-round reference implementation, kept for correctness checking of the cipher itself; it is NOT part of the attack pipeline. |
| 55 | +- `pairs` — differential `(c₀, c₁)` pairs used by `recover_prefix` / the neural distinguisher. |
| 56 | +- `sat_pairs` — known `(plaintext, ciphertext)` pairs used by the SAT suffix. |
99 | 57 |
|
100 | | -## Variable naming convention (critical when editing equations) |
| 58 | +On the CLI these are `--diff-pair` and `--sat-pair` respectively (both repeatable). Conflating them is a category error; early drafts of the test suite hit this and the fix was to split the API. |
101 | 59 |
|
102 | | -The ANF system uses three families of boolean variables. Keep them consistent across `sage-equations.py`, `polynomial-vars.py`, and `sage-CNF-convert.txt`: |
| 60 | +**Checkpoint policy.** `checkpoints/` is not committed by default. Produce a checkpoint with: |
103 | 61 |
|
104 | | -- `K0..K63` — the 64 key bits. Only these are the "unknowns" to recover. |
105 | | -- `L0..L191` — the NLFSR state bits across rounds. `L0..L31` is plaintext, `L32..L191` are intermediate state bits produced round by round. For 160 rounds the ciphertext sits at `L160..L191`. |
106 | | -- `A0..A159`, `B0..B159` — **linearization helper variables** introduced to keep each round equation at degree ≤ 2 for the SAT encoder. They represent the cubic/higher monomials of the KeeLoq NLF: `A_i = L_{i+31}·L_{i+26}`, `B_i = L_{i+31}·L_{i+1}`. The three equations per round (`eq1`, `eq2`, `eq3` in `sage-equations.py:31-33`) are (round update, A definition, B definition). Dropping or renaming A/B will change the degree and break the encoder. |
107 | | - |
108 | | -The core nonlinear function is defined identically in both cipher scripts: |
109 | | -`core(a,b,c,d,e) = d + e + ac + ae + bc + be + cd + de + ade + ace + abd + abc (mod 2)` |
110 | | - |
111 | | -## Running the pieces |
112 | | - |
113 | | -There is no build system, no test suite, and no linter config. Just Python 2 scripts and SageMath. Typical invocations: |
114 | | - |
115 | | -``` |
116 | | -python2 keeloq160-python.py # reference encrypt/decrypt of 160-round variant |
117 | | -python2 sage-equations.py # writes anf.txt |
118 | | -python2 polynomial-vars.py # writes vars.txt |
119 | | -sage sage-CNF-convert.txt # produces DIMACS CNF on stdout (see note) |
120 | | -minisat main160.cnf out.result # external solver |
121 | | -python2 parse-miniSAT.py # verifies recovered key against original |
122 | | -``` |
| 62 | + uv run keeloq neural auto --rounds 64 --trained-depth 56 \ |
| 63 | + --samples 10000000 --pairs 512 --checkpoint-out checkpoints/d64.pt |
123 | 64 |
|
124 | | -`sage-CNF-convert.txt` prints the CNF to stdout; the commented-out block at the end shows the original author's pattern for writing it to a file. The in-file comment warns "need to copy extra, doesn't output it all" — if output looks truncated, that is a known quirk, not a bug to chase. |
| 65 | +The regression test `tests/test_neural_e2e_64r.py` auto-skips when `checkpoints/d64.pt` is absent. The benchmark runner (`benchmarks/bench_neural.py`) reports `SKIP_MISSING_CHECKPOINT` rather than crashing on missing checkpoints — it's smoke-safe. |
125 | 66 |
|
126 | | -## Editing guidance specific to this repo |
| 67 | +## Red flags when editing |
127 | 68 |
|
128 | | -- **Round count is hard-coded in multiple places.** `sage-equations.py` loops `range(0,160)`, `polynomial-vars.py` sizes `A/B` to 160 and `L` to 192 (= 32 + 160). Changing the round count means updating all three places in lockstep or the SageMath ring declaration will mismatch the equations. |
129 | | -- **Key-bit hints are how the attack is tuned.** The README notes that without ~25–32 bits of key hinted into the system, miniSAT will return *some* satisfying assignment that is not the true key (underdetermined system). If you are changing the plaintext/ciphertext/key constants in `sage-equations.py`, also update the hint bits encoded via the `K_i + <bit>` terms in `sage-CNF-convert.txt` — the two files must describe the same instance. |
130 | | -- **`sage-equations.py` reverses the ciphertext list** (line 16) and indexes it as `L_{191-i} + ctext[31-i]` (line 24). This is deliberate bit-ordering, not a bug — preserve it on edits. |
131 | | -- **Decryption key index differs between the two cipher scripts.** `keeloq-python.py` (528 rounds) uses `k[15]` in `decroundfunction`; `keeloq160-python.py` (160 rounds) uses `k[31]`. This reflects the different residual key offset after each round count (528 = 8·64 + 16 vs. 160 = 2·64 + 32). Don't "unify" them. |
| 69 | +- Modifying anything in `legacy/`. Frozen; invocation wrappers live in `tests/compat_helpers.py`. |
| 70 | +- Using the PyTorch `.eval` method (the 4-letter inference-mode shorthand) anywhere in source or docs in this repo. A local security hook matches on its literal form (the 4 letters followed by a left paren) and blocks the write. Use `.train(False)` and `.train(True)` directly — `.eval` is just a one-line alias for `.train(False)`. |
| 71 | +- Changing the variable naming convention (`K{i}`, `L{j}_p{pair}`, `A{i}_p{pair}`, `B{i}_p{pair}`) without also updating the cross-validation tests. The compat test (`tests/test_compat.py::test_anf_matches_legacy_anf_txt`) canonicalizes away pair suffixes for comparison with the 2015 output; don't break that. |
| 72 | +- Committing checkpoints or `benchmark-results-neural/` artifacts. Both are gitignored. |
0 commit comments