Skip to content

docs: correct the document outline, and say what crawlers may do - #37

Merged
AviOfLagos merged 1 commit into
mainfrom
docs/semantics-and-crawlers
Aug 29, 2026
Merged

docs: correct the document outline, and say what crawlers may do#37
AviOfLagos merged 1 commit into
mainfrom
docs/semantics-and-crawlers

Conversation

@AviOfLagos

Copy link
Copy Markdown
Owner

Semantics

  • No <header> — the nav sat in no landmark, so landmark navigation gave you main and footer but nothing for the nav. Now wrapped.
  • Two pages skipped a heading level. check.html went h1 → h3 for the four checks, which are its top-level sections. changelog.html did the same, and worse: the release version was a <span>, so the Fixed/Added/Changed groups floated with no release heading above them — any tool building an outline saw eight sibling h3s belonging to nothing. Versions are h2 now, CSS mirrored so nothing moves visually.

All 9 pages: exactly one h1, no skipped levels, one main/header/footer.

Structured data

Only index.html had any. Now every page has a BreadcrumbList, plus:

page type why
infected.html HowTo, 4 steps with anchors "how do I remove this malware" is what gets typed mid-incident
install.html SoftwareApplication + version
check.html FAQPage the three questions people actually ask, including "does a clean process check mean I'm safe"

Crawlers

  • robots.txt names 29 crawlers explicitly — search engines and AI answering engines (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot…) rather than leaving them to infer permission from a bare wildcard, with a comment saying why.
  • llms.txt — compact authoritative summary: the threat, the response order, and the two things people reliably get wrong.
  • sitemap.xml gained lastmod.

Validated: 0 HTML parse errors, 0 invalid JSON-LD, sitemap parses as XML, snare scans itself clean, selftest 10/10.

Semantics
  The nav was not inside any landmark, so a reader jumping by landmark had a
  main and a footer but no header. It is now wrapped in <header>.

  Two pages skipped a heading level. check.html went h1 straight to h3 for the
  four checks, which ARE the page's top-level sections, so they are h2 now.
  changelog.html did the same, and worse: the release version was a <span>, so
  the "Fixed"/"Added"/"Changed" groups floated with no release heading above
  them and any tool building an outline saw eight sibling h3s belonging to
  nothing. Release versions are h2 now, the groups sit under them, and the CSS
  mirrors the old span rule so nothing moves visually.

  All nine pages: exactly one h1, no skipped levels, one main, one header, one
  footer.

Structured data
  Only index.html had any. Every page now carries a BreadcrumbList, and three
  carry the type that fits what someone is actually asking:

    infected.html  HowTo, four steps with anchors — "how do I remove this
                   malware" is what gets typed mid-incident
    install.html   SoftwareApplication with the current version
    check.html     FAQPage covering the three questions people ask: how do I
                   know, why did npm audit miss it, and does a clean process
                   check mean I am safe

Crawlers
  robots.txt named 29 crawlers explicitly rather than leaving search engines
  and AI answering engines to infer permission from a bare wildcard, and says
  in a comment why: the point of this site is that people find it before they
  are compromised.

  llms.txt gives an assistant a compact authoritative summary — the threat,
  the response order, and the two things people reliably get wrong — instead
  of leaving it to reconstruct one from nine pages of HTML.

  sitemap.xml gained lastmod, so a crawler can tell what changed.
@AviOfLagos
AviOfLagos merged commit 566727b into main Aug 29, 2026
2 checks passed
@AviOfLagos
AviOfLagos deleted the docs/semantics-and-crawlers branch August 29, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants