Skip to content

Commit 60b08bb

Browse files
authored
🐛 Settle an execution whose rendering the secret gate rejects (#573)
A rendered value the scanner detects left the execution abandoned: the rejected root Close(ok) still held its ordered append turn when durableRun appended the compensating Close(err) from the same scope, so completion never settled — and the credential had already left through the output stream before the journal gate ever saw it. The root-close appends now run inside their own scoped() boundary, so a rejected close returns its turn before the catch appends the failure close. The execution-owned output bridge screens each live chunk with the execution's scanner and withholds an uncleared chunk in full, while the journal gate remains the authority that fails the run; restored replay output bypasses the screen. Spec §8.1 records the observable rule. Claude-Session: https://claude.ai/code/session_01MNztk1fikXUnD2FW4yS6Kr Co-authored-by: Taras Mankovski <74687+taras@users.noreply.github.com>
1 parent 4515a90 commit 60b08bb

5 files changed

Lines changed: 222 additions & 5 deletions

File tree

‎packages/core/src/execute.ts‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ import { checkedFailureLedger } from "./component-failures.ts";
139139
import { provideTestHarnessInstallers } from "./test-harness.ts";
140140
import type { TestHarnessInstaller } from "./test-harness.ts";
141141
import type { CheckedFailures } from "./component-failures.ts";
142-
import { useSecretDetection } from "./secrets/policy.ts";
142+
import { clearsLiveOutput, useSecretDetection } from "./secrets/policy.ts";
143143
import { propsEnvironment } from "./eval-env.ts";
144144
import { liveEnvironment } from "./live-env.ts";
145145

@@ -1953,6 +1953,7 @@ function* executeDocument(
19531953
yield* spawn(function* () {
19541954
let emitted = false;
19551955
let emittedText = "";
1956+
let restoring = false;
19561957

19571958
// The ENTIRE setup and workflow sit inside one error boundary: once the
19581959
// handle exists, any failure closes output (with whatever rendered so
@@ -1965,8 +1966,15 @@ function* executeDocument(
19651966

19661967
// DocumentOutput → channel bridge (innermost middleware — output flows
19671968
// through caller-installed normalize/terminal middleware first, then here).
1969+
// A live chunk the execution's scanner cannot clear is withheld in full —
1970+
// not sent, accumulated, or described — while the durable journal gate
1971+
// stays the authority that fails the run. Restored output crossed that
1972+
// gate when it was journaled, so restoration is not screened again.
19681973
yield* DocumentOutput.around({
19691974
*output([text]) {
1975+
if (!restoring && !(yield* clearsLiveOutput(text))) {
1976+
return;
1977+
}
19701978
emitted = true;
19711979
emittedText += text;
19721980
yield* channel.send(text);
@@ -2186,7 +2194,9 @@ function* executeDocument(
21862194
// never the close value. A failed document takes the same path, so what
21872195
// it rendered first reaches consumers before its failure does.
21882196
if (!emitted && result.output) {
2197+
restoring = true;
21892198
yield* DocumentOutput.operations.output(result.output);
2199+
restoring = false;
21902200
}
21912201

21922202
yield* channel.close(result.output);

‎packages/core/src/secrets/policy.ts‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,37 @@ export function* scanSecrets(content: string): Operation<SecretFinding[]> {
213213
return yield* detection.scanner.scan(content);
214214
}
215215

216+
/**
217+
* Whether `content` may leave the execution as live output.
218+
*
219+
* An emit-or-withhold answer for the execution's own output bridge, and
220+
* nothing more: it makes no durable decision and supplies no error, because
221+
* the journal gate remains the authority that fails the run. Doubt withholds —
222+
* findings, a scanner that fails, and an absent or unauthentic policy all
223+
* answer `false`. Only an execution whose host explicitly disabled detection
224+
* clears everything, without a scanner existing to invoke.
225+
*
226+
* Package-private: the one caller is the output bridge `execute()` owns, and
227+
* exporting it publicly would advertise a screen whose answer is worth nothing
228+
* outside that bridge.
229+
*/
230+
export function* clearsLiveOutput(content: string): Operation<boolean> {
231+
let detection: ExecutionDetection;
232+
try {
233+
detection = yield* currentDetection();
234+
} catch {
235+
return false;
236+
}
237+
if (!(detection instanceof EnabledDetection)) {
238+
return true;
239+
}
240+
try {
241+
return (yield* detection.scanner.scan(content)).length === 0;
242+
} catch {
243+
return false;
244+
}
245+
}
246+
216247
/**
217248
* Apply the normalized policy to `stream`, and install it for this execution.
218249
*

‎packages/core/tests/secret-detection.test.ts‎

Lines changed: 163 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,18 @@
2323

2424
import { describe, it } from "@executablemd/test-support/bdd";
2525
import { expect } from "@executablemd/test-support/expect";
26-
import { all, createContext, Err, Ok, scoped, sleep, spawn, withResolvers } from "effection";
26+
import {
27+
all,
28+
createContext,
29+
ensure,
30+
Err,
31+
Ok,
32+
race,
33+
scoped,
34+
sleep,
35+
spawn,
36+
withResolvers,
37+
} from "effection";
2738
import type { Operation, Result } from "effection";
2839
import type { Json } from "../src/types.ts";
2940
import { forEach } from "@effectionx/stream-helpers";
@@ -49,6 +60,7 @@ import { inlineSource } from "../src/root-source.ts";
4960
import { registerComponents } from "../src/components/registration.ts";
5061
import { createScannerWith, createSecretScanner } from "../src/secrets/scanner.ts";
5162
import type { SecretScanner } from "../src/secrets/scanner.ts";
63+
import { SecretDetectedError } from "../src/secrets/findings.ts";
5264
import type { SecretFinding } from "../src/secrets/findings.ts";
5365
import {
5466
scanSecrets,
@@ -870,8 +882,158 @@ describe("default-on secret detection", () => {
870882
expect(replay.appendCount).toBe(0);
871883
});
872884
});
885+
886+
describe("output and settlement when a rendering is detected", () => {
887+
it("settles the clean control with its complete output", function* () {
888+
const run = yield* settledRendering("Token loopback-credential");
889+
890+
expect(run.result.ok).toBe(true);
891+
expect(run.chunks).toEqual([
892+
"# Probe\n\n",
893+
"\n\nThe header was Token loopback-credential.\n",
894+
]);
895+
expect(run.close).toBe("# Probe\n\n\n\nThe header was Token loopback-credential.\n");
896+
expect(run.events.map(label)).toEqual([
897+
"yield(import_component)",
898+
"yield(import_component)",
899+
"close(root)",
900+
]);
901+
const close = run.events.at(-1);
902+
expect(close?.type === "close" && close.result.status).toBe("ok");
903+
});
904+
905+
it("settles a detected rendering with Err and only safe partial output", function* () {
906+
const run = yield* settledRendering(CANARY);
907+
908+
expect(run.result.ok).toBe(false);
909+
const error = run.result.ok ? new Error("unreachable") : run.result.error;
910+
expect(error instanceof SecretDetectedError).toBe(true);
911+
912+
// The unsafe chunk is withheld in full; what closed is exactly the
913+
// cleared prefix the consumer already received.
914+
expect(run.chunks).toEqual(["# Probe\n\n"]);
915+
expect(run.close).toBe(run.chunks.join(""));
916+
917+
// The document's own source template legitimately persists with the root
918+
// import, so the sweep hunts the rendered sentence — the one carrying
919+
// the canary — rather than the template text.
920+
const tainted = `The header was ${CANARY}.`;
921+
922+
const journal = run.events.map(serializeDurableEvent).join("");
923+
const readable = [
924+
...run.chunks,
925+
run.close,
926+
String(error),
927+
error.name,
928+
error.message,
929+
error.stack ?? "",
930+
String(error.cause ?? ""),
931+
JSON.stringify(findingsOf(error)),
932+
journal,
933+
];
934+
for (const channel of readable) {
935+
expect(channel).not.toContain(CANARY);
936+
expect(channel).not.toContain(tainted);
937+
}
938+
939+
expect(run.events.map(label)).toEqual([
940+
"yield(import_component)",
941+
"yield(import_component)",
942+
"close(root)",
943+
]);
944+
const close = run.events.at(-1);
945+
expect(close?.type === "close" && close.result.status).toBe("err");
946+
expect(journal).toContain("SecretDetectedError");
947+
948+
expect(run.order).toEqual(["output-closed", "teardown", "completion-observed"]);
949+
});
950+
});
873951
});
874952

953+
/** A clean root that renders a registered component's value into its body. */
954+
const RENDERING = `# Probe
955+
956+
<Header as="header" />
957+
958+
The header was {header}.
959+
`;
960+
961+
/** What one drain-then-await run of the rendering probe left behind. */
962+
interface SettledRendering {
963+
chunks: string[];
964+
close: string;
965+
result: Result<Json>;
966+
events: DurableEvent[];
967+
order: string[];
968+
}
969+
970+
/**
971+
* Run the rendering probe end to end — output drained first, completion
972+
* awaited second — bounded by a failing five-second timer.
973+
*
974+
* The registered return value is the only variable, so what differs between
975+
* the clean control and the detected case is what detection did. A real
976+
* interval holds the event loop open for the whole sequence: an abandoned
977+
* completion under an idle loop is a test-runner diagnostic, while a held-open
978+
* loop turns it into an indefinite hang, and the losing timer turns either
979+
* into a plain failure.
980+
*/
981+
function* settledRendering(value: string): Operation<SettledRendering> {
982+
const backend = new InMemoryStream();
983+
const chunks: string[] = [];
984+
const order: string[] = [];
985+
986+
return yield* scoped(function* () {
987+
yield* registerComponents([
988+
{
989+
name: "Header",
990+
origin: "test:settlement",
991+
props: { type: "object", properties: {}, additionalProperties: false },
992+
returns: { type: "string" },
993+
// deno-lint-ignore require-yield
994+
*fn(): Operation<Json> {
995+
return value;
996+
},
997+
},
998+
]);
999+
yield* Execution.around({
1000+
*document([props], next) {
1001+
yield* ensure(() => {
1002+
order.push("teardown");
1003+
});
1004+
return yield* next(props);
1005+
},
1006+
});
1007+
1008+
const interval = setInterval(() => {}, 50);
1009+
yield* ensure(() => {
1010+
clearInterval(interval);
1011+
});
1012+
1013+
return yield* race([drainThenAwait(), timedOut()]);
1014+
});
1015+
1016+
function* drainThenAwait(): Operation<SettledRendering> {
1017+
const execution = yield* execute({ ...inlineSource(RENDERING), stream: backend });
1018+
const subscription = yield* execution.output;
1019+
let next = yield* subscription.next();
1020+
while (!next.done) {
1021+
chunks.push(next.value);
1022+
next = yield* subscription.next();
1023+
}
1024+
order.push("output-closed");
1025+
const close = next.value;
1026+
const result = yield* execution;
1027+
order.push("completion-observed");
1028+
return { chunks, close, result, events: backend.snapshot(), order };
1029+
}
1030+
1031+
function* timedOut(): Operation<never> {
1032+
yield* sleep(5000);
1033+
throw new Error("the drain-then-await sequence did not settle within five seconds");
1034+
}
1035+
}
1036+
8751037
/**
8761038
* Run a document and return what `observe` saw from inside the execution.
8771039
*

‎packages/durable-streams/run.ts‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
* See integration doc §10, protocol spec §4.
1414
*/
1515

16-
import { useScope } from "effection";
16+
import { scoped, useScope } from "effection";
1717
import type { Operation, Scope } from "effection";
1818
import { DurableContext } from "./context.ts";
1919
import { activeDurabilityFailure, appendDurableEvent } from "./durability.ts";
@@ -178,7 +178,12 @@ export function* durableRun<T extends WorkflowValue>(
178178
result: { status: "ok", value: result as Json },
179179
};
180180

181-
yield* appendDurableEvent(ctx, closeEvent);
181+
// The append's ordered turn is a resource of the scope it runs in, and
182+
// this frame survives a rejected close: the catch below appends the
183+
// compensating Close(err) from the same scope, so a turn still held here
184+
// would make that append wait forever. A boundary around the attempt
185+
// returns the turn as part of the outcome.
186+
yield* scoped(() => appendDurableEvent(ctx, closeEvent));
182187

183188
return result;
184189
} catch (error) {
@@ -207,7 +212,7 @@ export function* durableRun<T extends WorkflowValue>(
207212
};
208213

209214
try {
210-
yield* appendDurableEvent(ctx, closeEvent);
215+
yield* scoped(() => appendDurableEvent(ctx, closeEvent));
211216
} catch (closeError) {
212217
const closeDurabilityFailure = activeDurabilityFailure(ctx, closeError);
213218
if (closeDurabilityFailure) {

‎specs/executable-mdx-spec.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7469,6 +7469,15 @@ separate append that crosses the policy on its own — so a rejection never
74697469
implies an empty journal. There is no allowlist, sanitization, repair, or
74707470
approval: a finding is a code or data-flow defect to fix.
74717471

7472+
Rendered output is held to the same policy before it leaves the execution. A
7473+
live chunk the execution's scanner cannot clear — a finding, or a scan that
7474+
fails — is withheld from the output stream in full; earlier cleared chunks
7475+
remain observable and become the stream's partial close value. Withholding
7476+
decides nothing about the run: the journal gate remains the rejection
7477+
authority, and the rejected event stays absent. A replayed execution restores
7478+
its retained output without rescanning it — the journal it replays crossed the
7479+
gate when it was written.
7480+
74727481
Findings and scanner failures carry positions and rule identities, never the
74737482
matched value, the scanned content, or the detector's own error.
74747483

0 commit comments

Comments
 (0)