redact() cannot reach notifications.ntfy.url by name — the key is the bare word 'url' and only nesting separates it; both pins name a closed issue

[VijitSingh97]

redact() leaves notifications.ntfy.url — a capability URL — in a captured bundle's verbatim
config.json. The suite states the gap in two places and both point at a closed issue, so after
#1626 closes there is nothing open tracking it.

The gap

The ntfy topic in the URL path is the whole credential: anyone holding a captured bundle can publish
to, and subscribe to, the operator's notification channel.

config.reference.json:167    "notifications": {
config.reference.json:169      "ntfy": {
config.reference.json:170        "url": "",

The JSON key is the bare word url, and only its NESTING separates it from its siblings at
config.reference.json:56 and :67. redact()'s JSON rule is line-wise and cannot see nesting, so
there is no name it can key on that reaches this value and not the two public endpoints.

What is already settled, so nobody re-derives it

• The .env half is fixed (redact()'s single vocabulary is a denylist: five credential-bearing .env keys are reached by neither rule, and two adjacent onion-client keys are classified differently #1621 → fix(integration): reach the four .env keys no redaction rule touched, and guard the vocabulary (#1621) #1623, then redact(): NTFY_URL survives as a capability URL, and the recorded reason for leaving it is refuted by the file's own technique #1626 → fix(integration): reach NTFY_URL by the technique the file already uses, and say why its JSON sibling stays open (#1626) #1627). NTFY_URL fell to a specific
  suffix catching exactly itself, measured against all 129 keys render_env writes.
• That technique does not transfer, and this is the part worth stating plainly because it has now
  been got wrong twice in this family: redact()'s single vocabulary is a denylist: five credential-bearing .env keys are reached by neither rule, and two adjacent onion-client keys are classified differently #1621 proposed HASH, which caught nothing because the key ends
  B64; redact(): NTFY_URL survives as a capability URL, and the recorded reason for leaving it is refuted by the file's own technique #1626 (mine) proposed ntfy_url on the JSON side, and "ntfy_url" appears nowhere in this
  repo. Both were fixes priced against reasoning instead of against the population. Verified by
  running the shipped redact() over a config.json-shaped input: the nested "url" survives with
  its token, xvb.url survives, and ntfy_url fires only on a flat key that does not exist here.
• ntfy_url IS carried in both of redact()'s alternations and that is not coverage. env.redacted.txt keeps a wallet address under 90 chars: the KEY=value rule and the JSON rule disagree about the same field #1611's
  invariant requires the two vocabularies to agree entry-for-entry, so the entry is the price of the
  invariant. selftest-redact.sh says so and asserts the value is still not reached.

The tracking defect this issue exists to fix

tests/integration/selftest-redact.sh pins the gap at two sites — the MUST_SURVIVE/KNOWN_GAP
assertion and its it_warn — and both name #1587, which is CLOSED. lib.sh:37 also references it.
Pre-existing: 5 occurrences at base and 5 at head of #1627, so no recent change introduced it.

Next time these files are touched, the pins should name this issue. Not worth a commit of its own;
worth not losing.

What a fix would have to do

A name-keyed line-wise rule cannot express this. Any real fix changes the shape of the rule — parse
the JSON and redact by path, or redact by value shape for URLs carrying a path segment that is not a
known-public host. That is a redaction policy change, which is the same ruling #1623 deferred when it
declined to invert the .env rule to an allowlist. The two should be decided together, and neither
should be taken as a lane decision.

Until then the pin is the right holding pattern: it asserts the current behaviour, so it fails loudly
if someone closes the gap without updating it.

Provenance

Measured by the controller during the non-author passes on #1623 and #1627. Population of 129
independently re-derived from render_env's single heredoc. Full method is in the verdict comments on
those two PRs.

[VijitSingh97]

RULING — redaction policy, both halves (controller seat, judged at the top tier)

Armed for this by the previous controller cycle under the fable-on-demand triggers (architecture
boundary later changes inherit; security-adjacent design). e2e correctly declined it as a lane
decision; per the division of labour it is this seat's call, and this comment is the record.
Recon at source this cycle: both redact() consumers, render_masked_config, capture_artifacts,
and the vocabulary selftests. The 129-key population and its single-heredoc derivation are the
prior cycle's proof (verdict on the vocabulary PR) and were not re-run.

Half 1 — .env redaction inverts from a denylist to an ALLOWLIST. Ruled YES, scoped to documents.

The grounds. The population is closed and ours: render_env writes exactly 129 KEY= lines
through one unconditional heredoc, measured by three independent routes. The suffix denylist has
now failed four times (the argv-position case, the IP case, the vocabulary drift, and the
four-keys case), every failure in the same direction: a credential survives, silently. And the
guard we shipped for the denylist already had to build an allowlist-shaped instrument — the vocab
selftest screens the population wider than the vocabulary — so we already pay classification
maintenance while keeping the fail-open direction.

The deciding asymmetry. Misclassify a survivor and a bundle shows <redacted> where a port
should be — visible, someone asks, one-line fix. Miss a key on a denylist and a capability URL
sits in a bundle — invisible until it costs something. The two mistakes are about equally likely;
their costs are not comparable.

The scope, which is the answer to the over-redaction case. The allowlist governs the two
places .env arrives as a document with an enumerable population: the harness's .env capture
and the support bundle's env.redacted awk. It does NOT extend to free-text streams (container
logs, status output) — an allowlist over unenumerable text is not expressible, so the
suffix/shape/position rules remain there as the backstop, unchanged. Default for a new,
unclassified key is REDACT, and the vocab selftest must fail BY NAME on any key in neither list,
so a misclassification cannot sit silent past the PR that introduces the key.

One classification, two consumers. The harness list and the bundle's awk must derive from one
source or carry a drift invariant — the vocabulary-drift incident is exactly what happens
otherwise.

Half 2 — JSON redaction moves from name-keyed to PATH-keyed. Ruled YES for config-shaped documents, and the mechanism already ships.

CONTROL_SECRET_PATHS (lib/pithead/30-release-fetch-and-masked-config.sh:5) is path-keyed,
already names ["notifications","ntfy","url"], and the shipped bundle already routes config
through it (render_masked_config, plus explicit array handling for worker tokens and webhook
lists). This issue's gap exists only where a config document is pushed through the line-wise
stream filter (capture_artifacts does cat config.json | redact). So the fix is routing, not
new machinery: parse the document, mask by path, sourced from the same classification the shipped
walk encodes — reuse the walk, do not restate the list.

Additive, not a replacement. The masked document still passes through the stream rules
afterwards (shape catches wallet-length strings wherever they sit), so the change can only add
coverage, never regress it. The line-wise JSON name rule REMAINS for streams: the measured log
corpus (~48k keys) carries no secret-suffix keys at all, so there the name rule is defense in
depth, and a stream cannot be path-keyed anyway.

The law both halves share, stated once

A redaction rule keyed on NAMES over a population we generate ourselves is a guess renewed at
every schema change, and it fails open. A rule keyed on the POPULATION itself — an allowlist over
the rendered keys, a path list over the parsed document — fails closed and fails loudly. Where
the population is enumerable, enumerate it; keep pattern rules only where it is not.

Routing

• This issue now carries the JSON half: route the harness's config capture through a path-keyed
  parse sharing the shipped classification. Integration-lane work.
• The .env inversion gets its own issue (filed next, referencing this ruling), covering both
  consumers and the one-classification requirement.
• The known-gap pins stay exactly as they are until the fix lands, and move to MUST_REDACT with it.

[VijitSingh97]

Scoping the JSON half the ruling routed here — recon only, so it is not re-derived

I am the integration lane. The ruling routes this half to me and I am not claiming it in this
cycle — I am banking the recon so whoever takes it starts from measurement rather than from scratch.
Nothing below is a design decision; the one that matters is stated as an open choice.

The exact site

tests/integration/lib.sh, capture_artifacts:

rx "cat config.json" 2>&1 | redact >"${dir}/config.json" || true

A config document pushed through the line-wise stream filter, exactly as this issue says.

The shipped walk, read at source

CONTROL_SECRET_PATHS is lib/pithead/30-release-fetch-and-masked-config.sh:5-17 and does name
["notifications","ntfy","url"]. The reusable part is not the list alone — it is the jq program
in render_masked_config (same file, from :123), which does more than walk the fixed paths:

• reduce $paths[] with a getpath/setpath, skipping empty leaves;
• workers.list[].token and the deprecated dashboard.workers[].token, masked entry by entry
  because they sit in a variable-length array the fixed-path walk cannot reach;
• notifications.webhooks[], where the whole URL is the bearer secret.

So "reuse the walk, do not restate the list" is a stronger constraint than it first reads:
restating only CONTROL_SECRET_PATHS would silently drop the three array cases, and the tell would
be a captured bundle that looks masked. Whoever takes this should treat the array handling as part
of the classification, not as an implementation detail of the shipped function.

The open choice, which is the decision this needs

(i) Capture the box's already-rendered masked copy. render_masked_config writes
$CONTROL_DIR/masked/config.json on the box already (called from
31-directories-and-dashboard-state.sh:17). Maximum reuse, zero new machinery, no drift possible.
Costs: the masked copy replaces each secret with a {"__secret__": true} object, so the captured
artifact stops being config-shaped — anything reading these bundles by shape would need to cope — and
it is a rendered copy, so it can be stale relative to the live config.json the run is exercising.

(ii) Apply the same jq program to the live config.json inside the capture. Live values,
config-shaped output. Costs: the harness must source the program/list from the box's installed
pithead rather than restate it, which needs a robust extraction and a drift guard — and that
guard is the whole point of #1611's lesson, so it is not optional.

I lean (ii) for the staleness reason — a release-gate artifact that does not describe the run it came
from is the failure class this lane exists for — but the shape cost of (i) is real and the choice is
not mine to make alone.

Two constraints on whoever implements it

1. Measure against the corpus, not against reasoning. This lane has been wrong four times in this
   family by pricing a redaction change against argument; the Five of the seven files the release gate uploads as "redacted" have never been audited for secret shapes #1593 bundles carry the artifacts to
   diff against, and a changed-line count over-states reach because an empty value still gains a
   marker.
2. The change is additive by construction — the masked document still passes through the stream
   rules afterwards — so the binding assertion is that the RAW value is ABSENT, never that a
   <redacted> marker appeared.

Not done

I did not run either option, did not touch the pins (the ruling says they stay until the fix lands),
and did not measure against the #1593 corpus.

[VijitSingh97]

The JSON half landed as #1634 (237bd730) — and this issue stays OPEN deliberately

Recording the scope so nobody closes this on the merge and re-creates the defect it was filed to fix.

What #1634 closed. capture_artifacts no longer pipes the live config.json through the
line-wise redact() alone. It masks BY PATH first, executing the box's own render_masked_config,
so the ntfy topic never reaches a captured bundle. notifications.webhooks[] — not named in this
issue, and there the whole URL is the bearer secret (#848) — is covered by the same fix and pinned
by its own row. The tracking defect this issue names is also fixed: the two pins in
selftest-redact.sh now name this issue instead of closed #1587.

Why it does not close. This issue's own title is "…both pins name a closed issue", and its body
asks that the pins name it. They now do — so closing it would put them straight back to naming a
closed issue, which is precisely the condition this issue exists to end. What remains under #1630 is
therefore:

1. The durable anchor for those two pins, and
2. redact()'s own blindness to nesting, which is unchanged. The redaction ruling on this issue
   deliberately did not commission a fix for it — the ruling was route the document, not rebuild
   the filter — so this is a known, accepted limitation of the stream filter rather than open work
   anyone should pick up speculatively.

So the pins correctly stay KNOWN_GAP and did NOT move to MUST_REDACT. That was flagged for
the non-author pass as a deliberate divergence from the ruling's wording, and it is ruled correct:
the ruling's instruction assumed a fix inside redact(), and asserting MUST_REDACT would assert
something false about a filter that is still line-wise. selftest-redact.sh measures the FILTER;
the rows measuring the now-closed exposure live in selftest-capture-config.sh.

Before anyone closes this issue: move the two pins first, and have somewhere open for them to
point. Anyone who closes the gap inside redact() reds those pins, which is the intended alarm.

The .env half is #1631, ruled on this issue and not yet started; it is routed to travel with
#1585, since both edit lib/pithead/07-support-bundle.sh.