Add chaos rig + reject phone-number-shaped tokens as MACs - #13
Merged
Merged
Conversation
Adds tests/chaos_rig.py and tests/chaos_corpus.py: 67 messy real-world inputs (Windows ipconfig, Linux ip-link, Cisco show interfaces, BIOS labels, JSON/HTML/email pastes, wrappers, OCR typos, en-dashes, multi-MAC inputs, hex-word false positives, length boundaries) with a pinned expected outcome each. The rig classifies every result as match / match_ocr / partial / none, emits a Markdown report, and is wired into unittest via tests/test_chaos_rig.py so regressions fail CI. Running the rig surfaced one real defect: 1-800-555-1234 was cleaned into the fake MAC 18005551234 because the extractor only checked that the hex-only length sat in 6-12. Fixed by requiring pure-digit separator-bearing tokens to use a uniform group size of 2 or 4 chars (real MAC formats), which keeps 01:23:45:67:89:01 and 0123.4567.8901 working while rejecting 1/3/3/4 phone shapes. Same guard applied to web/app.js. The Python and JS smoke tests both pin the new behavior. tests/fixtures/chaos_report.md is the committed snapshot of the rig's output so reviewers can read the corpus and the behavior side by side. tests/fixtures/chaos_takeaways.md summarizes the higher-level findings.
StewAlexander-com
marked this pull request as ready for review
May 27, 2026 21:50
6 tasks
StewAlexander-com
added a commit
that referenced
this pull request
May 27, 2026
…prefixes (#14) PR #13 added a group-uniformity guard to the candidate scorer but lookup_detailed / macLookupDetailed had a separate fast path that fed normalize_mac(input) straight into the prefix matcher. Phone digits like '1-800-555-1234' survived normalize_mac (which only strips [-:.\s]) as 11 hex-valid chars, passed the 6-12 window, and surfaced as 'No registry entry for prefix 180055512' in the live PWA. This change adds _normalized_input_is_mac_shaped (Python) and normalizedInputIsMacShaped (JS) which apply the same group rule to the raw input before the fast path is allowed to set 'cleaned'. The rule is also broadened to accept group sizes in {2, 4, 6} (canonical 6x2, Cisco 3x4, half-and-half 6/6, truncated 4/2 like '0000.00'); tokens with 4 or more groups must additionally be uniformly sized so a UK phone like '+44-20-7946-0958' (2/2/4/4) still gets rejected. Tests: - chaos corpus picks up bare 1-800-555-1234, 555-1234, (415) 555-1212, +44-20-7946-0958, 192.168.1.1, 10.0.0.1 with expect=none, pinning cleaned=None so the live regression can't recur. - test_no_phone_numbers_become_macs now also asserts lookup_detailed(...).cleaned is None for each phone -- catching the actual UI-facing failure, not just extract_mac_candidates. - tests/web/mac_formats.mjs exports the new JS helpers and runs a simulated macLookupDetailed against phone and MAC inputs to assert the same outcome on the PWA side. - Regenerated tests/fixtures/chaos_report.md (73/73 cases pass) and updated tests/fixtures/chaos_takeaways.md to document both defects. Co-authored-by: Claude Code <claude-code@anthropic.com>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tests/chaos_rig.py+tests/chaos_corpus.py) throws 67 messy real-world inputs atmaclookup.lookup_detailedand classifies each result asmatch/match_ocr/partial/none. Covers Windows ipconfig, Linuxip link/ ifconfig, Cisco IOSshow interfaces+bia, Juniper / Arista / iDRAC / iLO labels, JSON / HTML / email / ticket prose, every wrapper and trailing punctuation combo, locally-administered / broadcast / multicast / all-zero edges, longest-prefix correctness, partial prefixes from 4-10 hex chars, OCR typo correction, hex-word false-positive guards (deadbeefcafe,face:beef:cafe,cisco,3com), Unicode-fancy punctuation (en-dashes, tabs, NBSP), multi-MAC inputs, and pathological empties / overlong runs.tests/fixtures/chaos_report.mdso reviewers can read corpus + behavior side by side without running the rig.tests/test_chaos_rig.pyso the existing CI step picks it up — total Python tests goes from 34 → 37.1-800-555-1234was being cleaned into the fake MAC18005551234. Reason: the candidate scorer only required that the hex-only length sit in 6-12 chars; pure-digit phone shapes slipped through. Fix inmaclookup.py:_has_macish_grouping(and mirrored inweb/app.js:hasMacishGrouping): when a separator-bearing token has no hex letters (a–f), require every separator-delimited group to be uniformly 2 or 4 chars. Keeps01:23:45:67:89:01(2/2/2/2/2/2) and0123.4567.8901(4/4/4) working; rejects1-800-555-1234(1/3/3/4) and(415) 555-1212-shaped strings.tests/fixtures/chaos_takeaways.mdrecords the higher-level findings: en-dashes accidentally reassemble, OCR I/l correction can legitimately end uppartial, hex blobs outside 6-12 chars are rejected outright, etc.Running the rig
```
python3 tests/chaos_rig.py # plain-text summary
python3 tests/chaos_rig.py --md # Markdown report to stdout
python3 tests/chaos_rig.py --strict # exits 1 on any mismatch
python3 tests/chaos_rig.py --md --out tests/fixtures/chaos_report.md
```
Test plan
🤖 Generated by Computer