Support encrypted DNS (DoH/DoT) and DNSSEC validation

[smolgroot]

Summary

Skypier Blackhole currently speaks plain DNS (UDP/TCP on port 53) on both the client-facing side and the upstream side. This issue proposes adding support for modern, secure DNS transports and validation:

1. DNS over TLS (DoT, RFC 7858) and DNS over HTTPS (DoH, RFC 8484)
2. DNSSEC validation (RFC 4033-4035)

Motivation

• Plain port 53 traffic is visible and tamperable by anyone on the path (ISP, hotspot, VPN exit). Encrypted upstreams keep queries private in transit, which matters for a resolver meant to run next to a VPN node.
• Modern clients (Android Private DNS, iOS/macOS profiles, Firefox/Chrome) increasingly expect DoT/DoH endpoints. Without a listener, those clients bypass the blackhole entirely, defeating the blocklist.
• DNSSEC validation lets us detect forged/poisoned upstream answers instead of blindly forwarding them.

Proposed scope

Phase 1: encrypted upstreams (client side of the forwarder)

• Allow configuring upstream resolvers as tls:// (853) and https:// URLs, e.g. tls://1.1.1.1, https://dns.quad9.net/dns-query.
• Hickory DNS (which we already use) supports this via the dns-over-rustls / dns-over-https-rustls feature flags, so this is mostly config plumbing plus TLS cert/SNI options.

Phase 2: encrypted listeners (server side)

• Optional DoT listener on 853 and DoH listener on 443 (configurable), with cert/key paths in the config file.
• Plain port 53 stays the default; encrypted listeners are opt-in.

Phase 3: DNSSEC

• Opt-in validating mode for forwarded queries (Hickory dnssec-ring feature): verify RRSIG chains against the root trust anchor, return SERVFAIL on bogus answers, and set the AD bit on validated ones.
• Needs care around blocked domains: our synthesized sinkhole answers are inherently unsigned, so we must not claim AD on them, and clients doing their own validation may reject them (worth documenting).

Non-goals (for now)

• DNS over QUIC (DoQ, RFC 9250) - can be a follow-up, Hickory has experimental support.
• Acting as an authoritative/signing server.

Key differences between the three, for reference

	DoT	DoH	DNSSEC
What it protects	Transport privacy/integrity	Transport privacy/integrity	Authenticity of the DNS data itself
Layer	TLS on dedicated port 853	HTTPS on port 443	Signatures inside DNS records, transport-agnostic
Hides queries from on-path observers	Yes	Yes (blends into HTTPS traffic)	No (plaintext unless combined with DoT/DoH)
Detects forged records	Only between the two endpoints	Only between the two endpoints	End-to-end, cryptographically

DoT/DoH and DNSSEC are complementary: the former encrypt the conversation, the latter proves the answers are genuine.