Skip to content

Commit 24397c0

Browse files
committed
fixes
1 parent f325931 commit 24397c0

19 files changed

Lines changed: 317 additions & 176 deletions

File tree

‎.pathloom-datasets/liquid-shadow/bundle/bundle.json‎

Lines changed: 4 additions & 4 deletions
Large diffs are not rendered by default.

‎.pathloom-datasets/liquid-shadow/bundle/report.json‎

Lines changed: 49 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -35,15 +35,23 @@
3535
"level": "high",
3636
"reasons": [
3737
"tool exposure is observational rather than randomized",
38-
"actors who use a tool may differ from the control cohort before exposure"
38+
"actors who use a tool may differ from the control cohort before exposure",
39+
"invoked-at gaps are descriptive timing only, not elapsed-time causal windows"
3940
]
4041
},
4142
"interpretation": "linked_cohort_association_only",
4243
"returnWindow": {
44+
"aggregate": {
45+
"immediateNextSessionReturns": 6,
46+
"medianInvokedAtGapToReturn": 1135,
47+
"returningActorCount": 6,
48+
"sustainedMultiSessionReturns": 0
49+
},
50+
"claimBoundary": "not_incremental_effect_not_causal",
4351
"kind": "next_observed_session",
4452
"label": "second_observed_session",
45-
"timingMode": "session_count_only",
46-
"explanation": "Return currently means the actor had any later observed session. Pathloom does not yet model elapsed time, censoring, or intervention timing."
53+
"timingMode": "session_order_plus_invoked_at_gap",
54+
"explanation": "Return means the actor had a later observed session. Pathloom also reports immediate (exactly two sessions) vs sustained (three or more) return shapes and median invoked-at gap between first and second sessions when timestamps exist. This is still observational — not incremental lift or causal timing."
4755
},
4856
"signalTaxonomy": {
4957
"candidate": "candidate_return_correlate",
@@ -58,6 +66,23 @@
5866
"id": "activation_tool_report",
5967
"items": [
6068
{
69+
"cohortContext": {
70+
"cohortSemantics": "first_session_exposure_vs_linked_control_without_exposure",
71+
"comparisonId": "activation:shadow_ops_context:first_session_exposure",
72+
"controlActorCount": 6,
73+
"controlCohortLabel": "linked actors without shadow_ops_context in their first observed session",
74+
"controlReturnRate": 0,
75+
"exposedActorCount": 6,
76+
"exposureCohortLabel": "linked actors who invoked shadow_ops_context in their first observed session",
77+
"exposedReturnRate": 1,
78+
"returnWindow": {
79+
"immediateNextSessionReturns": 6,
80+
"kind": "next_observed_session",
81+
"label": "second_observed_session",
82+
"medianInvokedAtGapToReturn": 1135,
83+
"sustainedMultiSessionReturns": 0
84+
}
85+
},
6186
"confidenceBand": {
6287
"controlLower": 0,
6388
"controlUpper": 0.39,
@@ -201,7 +226,16 @@
201226
"sequenceLabel": "shadow_search_path -> shadow_inspect_file",
202227
"sessionCount": 6,
203228
"terminalToolBaselineErrorRate": 0,
204-
"terminalToolErrorLift": 1
229+
"terminalToolErrorLift": 1,
230+
"trajectoryContext": {
231+
"cohortSemantics": "ordered_paths_sharing_terminal_tool_and_suffix_peer_bucket_excludes_focus_path",
232+
"familyDistinctPathCount": 2,
233+
"familyId": "terminal:shadow_inspect_file:suffix_tail:shadow_inspect_file:len:2:risky_sequence",
234+
"familyLabel": "other routes into shadow_inspect_file",
235+
"peerObservationCount": 6,
236+
"suffixBucketObservationCount": 12,
237+
"windowLength": 2
238+
}
205239
},
206240
{
207241
"baselineKind": "matched_suffix_peers",
@@ -252,7 +286,16 @@
252286
"successLift": 1,
253287
"successRate": 1,
254288
"terminalToolBaselineSuccessRate": 0,
255-
"terminalToolSuccessLift": 1
289+
"terminalToolSuccessLift": 1,
290+
"trajectoryContext": {
291+
"cohortSemantics": "ordered_paths_sharing_terminal_tool_and_suffix_peer_bucket_excludes_focus_path",
292+
"familyDistinctPathCount": 2,
293+
"familyId": "terminal:shadow_inspect_file:suffix_tail:shadow_inspect_file:len:2:golden_path",
294+
"familyLabel": "other routes into shadow_inspect_file",
295+
"peerObservationCount": 6,
296+
"suffixBucketObservationCount": 12,
297+
"windowLength": 2
298+
}
256299
}
257300
],
258301
"recommendation": "Focus on the highest-lift failure endpoint first. Tighten the handoff into that terminal tool or add a safer bridge before the failing step.",
@@ -428,7 +471,7 @@
428471
}
429472
}
430473
],
431-
"generatedAt": "2026-05-27T08:45:52.841Z",
474+
"generatedAt": "2026-05-29T07:40:46.792Z",
432475
"reportVersion": "1.0",
433476
"sourceKey": "logfile:liquid-shadow-dogfood",
434477
"summary": {

‎.pathloom-datasets/liquid-shadow/bundle/report.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Pathloom Report
22

33
Source: `logfile:liquid-shadow-dogfood`
4-
Generated: 2026-05-27T08:45:52.841Z
4+
Generated: 2026-05-29T07:40:46.792Z
55
Sessions: 18
66
Events: 42
77
Catalog tools: 6
@@ -16,12 +16,12 @@ Evidence strength: **Credible finding** (full claim)
1616
The dataset supported the full claim, and the observed pattern cleared the emission threshold.
1717

1818
Interpretation limits:
19-
- Linked-cohort association only: Pathloom is not estimating incremental effect or causal lift here.
20-
- Return window: second_observed_session (Return currently means the actor had any later observed session. Pathloom does not yet model elapsed time, censoring, or intervention timing.)
21-
- Confounding risk: high; tool exposure is observational rather than randomized; actors who use a tool may differ from the control cohort before exposure.
19+
- Linked-cohort association only: Pathloom is not estimating incremental effect or causal lift here (not_incremental_effect_not_causal).
20+
- Return window: second_observed_session (session_order_plus_invoked_at_gap). Return means the actor had a later observed session. Pathloom also reports immediate (exactly two sessions) vs sustained (three or more) return shapes and median invoked-at gap between first and second sessions when timestamps exist. This is still observational — not incremental lift or causal timing. Aggregate timing: 6 returning actors (6 immediate next-session, 0 sustained multi-session; median invoked-at gap 1135).
21+
- Confounding risk: high; tool exposure is observational rather than randomized; actors who use a tool may differ from the control cohort before exposure; invoked-at gaps are descriptive timing only, not elapsed-time causal windows.
2222
- Actor linkage gate: 12 linked actors satisfied the privacy-safe cohort requirement.
2323

24-
- `shadow_ops_context` is a credible cohort signal: 6/6 linked actors who used it in session 1 returned for a second observed session (100%) versus 0% for the linked control cohort with no control-group returns observed. 95% interval 61-100% vs control 0-39%
24+
- `shadow_ops_context` is a credible cohort signal: 6/6 linked actors who used it in session 1 returned for a second observed session (100%) versus 0% for the linked control cohort with no control-group returns observed. 95% interval 61-100% vs control 0-39% Cohort `activation:shadow_ops_context:first_session_exposure`: linked actors who invoked shadow_ops_context in their first observed session vs linked actors without shadow_ops_context in their first observed session (6 immediate next-session returns, 0 sustained multi-session returns; median invoked-at gap to return 1135).
2525

2626
Recommendation: Surface this tool earlier in onboarding and make its value legible, because it is the strongest confidence-backed cohort return signal in this dataset.
2727

@@ -46,8 +46,8 @@ Evidence strength: **Credible finding** (full claim)
4646

4747
The dataset supported the full claim, and the observed pattern cleared the emission threshold.
4848

49-
- `shadow_search_path -> shadow_inspect_file` reliably precedes the `shadow_inspect_file` failure endpoint: 100% errors across 6 repeated paths. other routes into shadow_inspect_file fail 0% across 6 peer paths, and other routes ending at shadow_inspect_file fail 0%; seen in cursor only.
50-
- `shadow_search_concept -> shadow_inspect_file` is a reusable success path into `shadow_inspect_file`: 100% success across 6 repeated paths. other routes into shadow_inspect_file succeed 0% across 6 peer paths, and other routes ending at shadow_inspect_file succeed 0%; seen in claude-desktop only.
49+
- `shadow_search_path -> shadow_inspect_file` reliably precedes the `shadow_inspect_file` failure endpoint: 100% errors across 6 repeated paths. other routes into shadow_inspect_file fail 0% across 6 peer paths, and other routes ending at shadow_inspect_file fail 0%; seen in cursor only. Trajectory family `terminal:shadow_inspect_file:suffix_tail:shadow_inspect_file:len:2:risky_sequence` (2-hop): other routes into shadow_inspect_file; compared vs 6 suffix-peer observations (ordered paths sharing terminal tool and suffix peer bucket excludes focus path; 2 distinct paths in the peer bucket).
50+
- `shadow_search_concept -> shadow_inspect_file` is a reusable success path into `shadow_inspect_file`: 100% success across 6 repeated paths. other routes into shadow_inspect_file succeed 0% across 6 peer paths, and other routes ending at shadow_inspect_file succeed 0%; seen in claude-desktop only. Trajectory family `terminal:shadow_inspect_file:suffix_tail:shadow_inspect_file:len:2:golden_path` (2-hop): other routes into shadow_inspect_file; compared vs 6 suffix-peer observations (ordered paths sharing terminal tool and suffix peer bucket excludes focus path; 2 distinct paths in the peer bucket).
5151

5252
Recommendation: Focus on the highest-lift failure endpoint first. Tighten the handoff into that terminal tool or add a safer bridge before the failing step.
5353

‎.pathloom-datasets/liquid-shadow/bundle/share-summary.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
6 ready findings, 0 clear findings, 0 suppressed findings, for logfile:liquid-shadow-dogfood across 18 sessions.
44

55
Source: `logfile:liquid-shadow-dogfood`
6-
Generated: 2026-05-27T08:45:52.841Z
6+
Generated: 2026-05-29T07:40:46.792Z
77
Telemetry: sessions explicit_session_id; trace none; span lineage none; first-class fields none
88

99
## Top findings

‎.pathloom-datasets/liquid-shadow/feedback.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
# Pathloom Feedback Review
22

3-
0 of 11 feedback targets reviewed for snapshot logfile:liquid-shadow-dogfood@20260527T084552878Z.
3+
0 of 11 feedback targets reviewed for snapshot logfile:liquid-shadow-dogfood@20260529T074046834Z.
44

55
Source: `logfile:liquid-shadow-dogfood`
6-
Snapshot: `logfile:liquid-shadow-dogfood@20260527T084552878Z`
7-
Captured: 2026-05-27T08:45:52.878Z
6+
Snapshot: `logfile:liquid-shadow-dogfood@20260529T074046834Z`
7+
Captured: 2026-05-29T07:40:46.834Z
88

99
## Summary
1010

0 commit comments

Comments
 (0)