wuzzy/contracts/canonicalize-v1.feature at master · Memetic-Block/wuzzy · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
@t1 @protocol-v1
Feature: wuzzy/crawl canonicalization protocol v1
  The canonicalizer is the pinned public procedure that every EAS attestation's
  protocolVersion=1 refers to. These scenarios + the fixture vectors in
  fixtures/canonicalize-v1/ ARE the protocol spec in executable form; VERIFY.md
  is its prose twin. Once the first mainnet attestation lands, this feature and
  its fixtures are FROZEN — behavior changes are protocol v2 in a new feature.

  Background:
    Given the canonicalizer implementation for protocol "wuzzy/crawl" version 1

  Scenario Outline: conformance vectors reproduce pinned markdown and hash
    Given fixture input "<input>"
    When it is canonicalized under protocol v1
    Then the canonical markdown equals fixture "<expected_md>"
    And the sha256 content hash equals the value in "<expected_hash>"

    Examples:
      | input                      | expected_md                   | expected_hash                   |
      | docs-page.html             | docs-page.md                  | docs-page.hash                  |
      | code-blocks.html           | code-blocks.md                | code-blocks.hash                |
      | nested-lists.html          | nested-lists.md               | nested-lists.hash               |
      | unicode-nfc.html           | unicode-nfc.md                | unicode-nfc.hash                |
      | crlf-endings.html          | crlf-endings.md               | crlf-endings.hash               |
      | trailing-whitespace.html   | trailing-whitespace.md        | trailing-whitespace.hash        |
      | markdown-native.md         | markdown-native.expected.md   | markdown-native.hash            |
      | readability-fallback.html  | readability-fallback.md       | readability-fallback.hash       |

  Scenario: scripts, styles and comments never reach the hash
    Given fixture input "inline-code.html" with an inline script, a style block and comments
    When it is canonicalized under protocol v1
    Then the canonical markdown contains none of their text
    And it contains the article body

  Scenario: normalization is idempotent
    Given any canonical markdown output from protocol v1
    When it is normalized again under protocol v1
    Then the output is byte-identical
    And the content hash is unchanged

  Scenario: thin pages are rejected, not hashed
    Given fixture input "thin-page.html" whose extracted content is under 80 characters
    When it is canonicalized under protocol v1
    Then the pipeline marks the page as skipped
    And no document row or attestation is produced for it

  Scenario: site chrome never reaches the hash
    Given fixture input "docs-page.html" containing nav, sidebar and footer text
    When it is canonicalized under protocol v1
    Then the canonical markdown contains none of that text
    And it contains the article body

  Scenario: the fetch URL is part of the input
    Given fixture input "relative-links.html" canonicalized as fetched from two different URLs
    When relative links in the page resolve against each URL
    Then the two canonical outputs differ
    And each is stable across repeated runs at the same URL

  Scenario: line endings do not move the content hash
    Given fixture input "docs-page.html"
    And the same bytes with every LF replaced by CRLF
    When both are canonicalized under protocol v1
    Then the content hashes are equal
    And the raw hashes differ

  Scenario: malformed markup canonicalizes without failing
    Given fixture input "malformed.html" with unclosed and mis-nested tags
    When it is canonicalized under protocol v1
    Then a canonical markdown output is produced
    And canonicalizing it a second time yields the identical hash

  Scenario: invalid UTF-8 bytes decode deterministically
    Given fixture input "invalid-utf8.html" containing a truncated multi-byte sequence
    When it is canonicalized under protocol v1
    Then the invalid sequence becomes U+FFFD in the canonical markdown
    And the content hash is the same on every run

  Scenario: raw hash commits to exact received bytes
    Given fixture input "docs-page.html" as a byte stream
    When the raw hash is computed
    Then it equals sha256 over those exact bytes with no normalization applied