Skip to content

Commit cdda9b3

Browse files
skosovskyskosovsky
committed
feat: boundary contracts
1 parent fa33637 commit cdda9b3

20 files changed

Lines changed: 2343 additions & 213 deletions

‎README.md‎

Lines changed: 109 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
[![Go Report Card](https://goreportcard.com/badge/github.com/skosovsky/contexty)](https://goreportcard.com/report/github.com/skosovsky/contexty)
55
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
66

7-
`contexty` is a **semantic context engine** for LLM applications: typed message AST, actor-aware provider-role projection, typed tool payloads, context artifacts, immutable conversation deltas, named views (`RenderView`), unified budgeting, and `Compile()` → `CompileResult` (payload + immutable `Source` + `Introduced` + `DerivePersistenceProjection`).
7+
`contexty` is a **semantic context engine** for LLM applications: typed message AST, actor-aware provider-role projection, typed current turns, durable identity policies, typed tool payloads, typed context artifacts, immutable conversation deltas, named compile targets, unified budgeting, and `Compile()` → `CompileResult` (payload + immutable `Source` + `NormalizedSnapshot` + `Writeback` + `Projections` + `DerivePersistenceProjection`).
88

99
## Installation
1010

@@ -44,13 +44,24 @@ engine := contexty.NewEngine(
4444
)),
4545
)
4646

47+
turn := contexty.NewCurrentTurn(
48+
contexty.TextMessage(contexty.RoleUser, "Current turn"),
49+
).WithPromptSafe(contexty.TextMessage(contexty.RoleUser, "Current turn, redacted for prompt"))
50+
4751
result, err := engine.Compile(ctx, contexty.CompileRequest{
48-
Pending: []contexty.Message{contexty.TextMessage(contexty.RoleUser, "Current turn")},
52+
CurrentTurn: &turn,
53+
IdentityPolicy: contexty.NewStableMessageIdentityPolicy("chat"),
54+
RequireDurableIdentity: true,
4955
Options: []contexty.CompileOption{
5056
contexty.WithResolveVar("locale", "en-US"),
5157
},
58+
Targets: []contexty.CompileTarget{{
59+
Name: "classifier_history",
60+
SourceSegment: contexty.SegmentHistory,
61+
}},
5262
})
5363
toSave := result.DerivePersistenceProjection(contexty.SegmentHistory)
64+
_ = result.Writeback // assigned durable IDs and normalized snapshot
5465
_ = store.ApplyDelta(ctx, "chat-1", 2, contexty.ConversationDelta{
5566
Operation: contexty.DeltaReplaceSegment,
5667
Segment: contexty.SegmentHistory,
@@ -62,41 +73,44 @@ _ = store.ApplyDelta(ctx, "chat-1", 2, contexty.ConversationDelta{
6273

6374
```go
6475
result, err := engine.Compile(ctx, contexty.CompileRequest{
65-
System: systemMsgs,
66-
History: historyMsgs,
67-
Memory: memoryMsgs,
68-
Tools: toolMsgs,
69-
Pending: pendingMsgs,
70-
Options: []contexty.CompileOption{
71-
contexty.WithEphemeralPatch(contexty.MessageSelector{
72-
Segment: contexty.SegmentHistory, Role: contexty.RoleUser, Position: contexty.PositionLast,
73-
}, "REDACTED"),
74-
contexty.WithResolveVar("locale", "ru-RU"),
75-
},
76+
System: systemMsgs,
77+
History: historyMsgs,
78+
Memory: memoryMsgs,
79+
Tools: toolMsgs,
80+
CurrentTurn: &currentTurn,
81+
IdentityPolicy: contexty.NewStableMessageIdentityPolicy("chat"),
82+
RequireDurableIdentity: true,
83+
Targets: []contexty.CompileTarget{{
84+
Name: "classifier_history",
85+
SourceSegment: contexty.SegmentHistory,
86+
Budget: classifierBudgetPipe,
87+
}},
88+
Options: []contexty.CompileOption{contexty.WithResolveVar("locale", "ru-RU")},
7689
})
7790
payload := result.Payload
7891
toSave := result.DerivePersistenceProjection(contexty.SegmentHistory)
92+
classifier := result.Projections["classifier_history"]
7993
```
8094

81-
`CompileResult.Source` is an immutable freeze of input messages (before pipeline mutations). `CompileResult.Introduced` captures pre-transform baselines for payload-born IDs (registered post-deferred, before hooks/patches). Use `DerivePersistenceProjection` for checkpoint persistence instead of parsing `Transformations`.
95+
`CompileResult.Source` is an immutable freeze of normalized input messages (before pipeline mutations). `CompileResult.NormalizedSnapshot` and `CompileResult.Writeback` expose durable ID normalization and checkpoint writeback intent. `CompileResult.Introduced` captures pre-transform baselines for payload-born IDs (registered post-deferred, before hooks/patches). Use `DerivePersistenceProjection` for checkpoint persistence instead of parsing `Transformations`.
8296

83-
**Pipeline order:** freeze Source → deferred → ephemeral patches (pre-budget) → hooks → segment formatters → budget preflight → budget(history) → ephemeral patches (post-budget, history + Pending) → payload.
97+
**Pipeline order:** normalize IDs/current turn → freeze Source → deferred → low-level ephemeral patches (pre-budget) → hooks → segment formatters → budget preflight → budget(history + protected current turn) → low-level ephemeral patches (post-budget) → payload → named compile targets.
8498

85-
The current clear-break contract is summarized below; the task-level implementation spec is `.cursor/docs/task15.md`.
99+
The current clear-break contract is summarized below; the task-level implementation spec is `.cursor/docs/task16.md`.
86100

87101
### Migrating from Task13
88102

89103
1. Removed prompt-origin aliases now map to `Origin` / `TemplateID`.
90-
2. Replace `WithOverlay` with `CompileRequest.Options` (`WithResolveVar`, `WithEphemeralPatch`).
91-
3. Use `RenderView` + `WithNamedView` for classifier projections.
104+
2. Replace `WithOverlay` with `CompileRequest.Options` for resolve vars and `CompileRequest.CurrentTurn` for prompt-only current-turn projection.
105+
3. Use `CompileRequest.Targets` and `CompileResult.Projections` for classifier projections built from the same compile pass.
92106
4. Set `DeferredBlock.MergePolicy` for origin/layer collision handling.
93107
5. Persist with `DerivePersistenceProjection`.
94108

95109
### Migrating from Task12
96110

97111
1. Use `CompileRequest` / `CompileResult` instead of snapshot-only compile and `AbstractPayload`.
98112
2. Set `Message.ID` as the semantic node ID; use `SourceRefs` for external identity and typed `Extensions` for host metadata.
99-
3. Put the current turn in `Pending`, not post-compile append.
113+
3. Put the active user input in `CurrentTurn`; reserve `Pending` for low-level protected pending messages.
100114
4. Pass `Tools` explicitly when needed.
101115
5. Inspect `result.Transformations[msgID]` instead of string diffs on payload.
102116

@@ -108,39 +122,47 @@ engine := contexty.NewEngine(
108122
contexty.WithBudgetPipeline(contexty.SegmentHistory, pipe),
109123
)
110124
result, _ := engine.CompileSnapshot(ctx, contexty.CompileRequest{
111-
History: msgs,
112-
Pending: []contexty.Message{currentTurn},
125+
History: msgs,
126+
CurrentTurn: &currentTurn,
127+
IdentityPolicy: contexty.NewStableMessageIdentityPolicy("snapshot"),
128+
RequireDurableIdentity: true,
113129
})
114130
```
115131

116132
Observer telemetry (`WithObserver`, `WithBudgetObserver`) behaves the same on `Compile` and `CompileSnapshot`.
117133

118-
## Views (non-mutating render)
134+
## Named Compile Targets
119135

120-
Built-in views render all segments (system → history → tools → memory):
136+
Use compile targets when a classifier, router, evaluator, or secondary provider needs a projection from the same normalized snapshot, current turn, artifacts, transforms, and budgeted history:
121137

122138
```go
123-
snap, _ := store.LoadState(ctx, "chat-1")
124-
engine := contexty.NewEngine()
125-
xml, _ := engine.RenderView(ctx, snap, string(contexty.ViewLLMXML))
126-
flat, _ := contexty.Render(ctx, snap, contexty.ViewFlatClassifier) // shortcut: NewEngine() + builtin RenderView only
139+
result, _ := engine.Compile(ctx, contexty.CompileRequest{
140+
CurrentTurn: &currentTurn,
141+
Targets: []contexty.CompileTarget{{
142+
Name: "classifier_history",
143+
SourceSegment: contexty.SegmentHistory,
144+
Budget: classifierBudgetPipe,
145+
}},
146+
})
147+
classifier := result.Projections["classifier_history"]
148+
_ = classifier.Text
127149
```
128150

129-
Custom named views with budget/formatter:
151+
`CompileProjection` includes rendered `Text`, typed `Messages`, transform records, participating artifact IDs, frozen `Source` (normalized request before pipeline mutations), and `InputSnapshot` (the compiled snapshot used as the target input). A target `View` is a built-in rendered view and is mutually exclusive with `SourceSegment`, `Budget`, and `Formatter`; use segment targets when target-local budget or formatting is required.
152+
153+
## Views (non-mutating render)
154+
155+
`Render` / `RenderView` remain available for read-only snapshot inspection. They do not run the compile pipeline, do not apply transform hooks, and do not see `CurrentTurn`.
156+
157+
Built-in views render all stored segments (system → history → tools → memory):
130158

131159
```go
132-
engine := contexty.NewEngine(
133-
contexty.WithNamedView("classifier", contexty.ViewConfiguration{
134-
SourceSegment: contexty.SegmentHistory,
135-
Budget: classifierBudgetPipe,
136-
}),
137-
)
138-
out, _ := engine.RenderView(ctx, snap, "classifier")
160+
snap, _ := store.LoadState(ctx, "chat-1")
161+
xml, _ := contexty.Render(ctx, snap, contexty.ViewLLMXML)
162+
flat, _ := contexty.Render(ctx, snap, contexty.ViewFlatClassifier)
139163
```
140164

141-
`Render` / `RenderView` never mutate the input snapshot. They do **not** apply transform hooks — use `Engine.Compile()` when you need redaction or truncation before sending to an LLM.
142-
143-
Built-in view names (`llm_xml`, `flat_classifier`) are resolved before the custom registry; `WithNamedView("llm_xml", …)` does not override the built-in formatter. Custom views join segment messages as plain text (not LLMXML).
165+
For classifier/router/evaluator projections, prefer named compile targets. `RenderView` is for already-materialized snapshot inspection and legacy callers.
144166

145167
## Messages, Actors, and Source Refs
146168

@@ -248,22 +270,32 @@ _ = result.Introduced // pre-transform baselines for payload-born IDs
248270

249271
Patches and `Pending` are compile-only. `DerivePersistenceProjection` excludes evicted/truncated messages and returns Source originals for formatted messages. If `Pending` alone exceeds `TokenLimit`, compile returns `ErrPendingExceedsBudget`.
250272

251-
## Ephemeral patches
273+
## Current Turn and Identity
274+
275+
Use `CurrentTurn` for active input that needs different provider-facing and checkpoint-facing representations:
252276

253277
```go
254-
result, _ := engine.Compile(ctx, contexty.CompileRequest{
255-
Pending: []contexty.Message{currentUserTurn},
256-
Options: []contexty.CompileOption{
257-
contexty.WithEphemeralPatch(contexty.MessageSelector{
258-
Segment: contexty.SegmentHistory,
259-
Role: contexty.RoleUser,
260-
Position: contexty.PositionLast,
261-
}, "REDACTED"),
262-
},
278+
turn := contexty.NewCurrentTurn(contexty.TextMessage(contexty.RoleUser, "raw input")).
279+
WithPromptSafe(contexty.TextMessage(contexty.RoleUser, "redacted input")).
280+
WithPersistence(contexty.CurrentTurnPersistRaw)
281+
282+
result, err := engine.Compile(ctx, contexty.CompileRequest{
283+
CurrentTurn: &turn,
284+
IdentityPolicy: contexty.NewStableMessageIdentityPolicy("chat"),
285+
RequireDurableIdentity: true,
263286
})
287+
_ = result.Writeback.Snapshot
288+
_ = result.Writeback.Messages
289+
_ = err
264290
```
265291

266-
`MessageSelector.Position`: zero value is `PositionFirst`; an unrecognized value defaults to `PositionLast`. Pre-budget patches apply to non-history segments; post-budget patches apply to history (including merged `Pending`).
292+
`CurrentTurnPersistRaw` stores the original input, `CurrentTurnPersistPromptSafe` stores the prompt-safe representation, and `CurrentTurnPersistNone` skips current-turn checkpoint persistence. When `RequireDurableIdentity` is true, missing message IDs require an explicit `IdentityPolicy`; otherwise compile fails with `ErrMissingIdentityPolicy`.
293+
294+
## Low-Level Ephemeral Patches
295+
296+
Prefer `CurrentTurn` for prompt-only current-turn redaction. `WithEphemeralPatch` remains a low-level escape hatch for compile-only replacement of already-addressable messages in internal pipelines. It is not the durable current-turn contract.
297+
298+
Pre-budget patches apply to non-history segments; post-budget patches apply to history.
267299

268300
## Segment formatters
269301

@@ -308,7 +340,7 @@ engine := contexty.NewEngine(
308340

309341
Tool-call turns are truncated atomically by default (`KeepTurnAtomicity` defaults to `true`). Setting `KeepTurnAtomicity` to `false` enables fast-path index truncation at the strategy level; `BudgetPipeline` still repairs orphan tool pairs via `enforceToolPairAtomicity`.
310342

311-
When using a custom `Summarizer`, return a summary with a **new** `Message.ID`. Reusing a truncated message ID prevents the summary from appearing in `DerivePersistenceProjection`.
343+
When using a custom `Summarizer`, do not reuse a truncated message ID for the summary. In durable compile flows, leave the summary ID empty and let `IdentityPolicy` assign it.
312344

313345
**Canonical tool-turn layout** for atomic truncation: `RoleAssistant` with `ToolCallPart`(s), then `RoleTool` message(s) with matching `ToolResultPart.ToolCallID`. Use `ToolRoundFromMessages` / `ToolRound.Validate` for first-class validation. `ToolTurnUsesCanonicalLayout` remains a lightweight layout predicate.
314346

@@ -335,7 +367,33 @@ _ = result
335367
_ = err
336368
```
337369

338-
Turn-bound retrieval artifacts are visible only when `CompileRequest.TurnID` matches `BoundTurnID`. Ownership is `OwnerRef`, a typed `SourceRef` owned by the host application. Ephemeral artifacts and `ArtifactPersistenceSkip` are omitted from checkpoints; `ArtifactPersistenceStore` forces checkpoint persistence.
370+
Turn-bound retrieval artifacts are visible only when `CompileRequest.TurnID` matches `BoundTurnID`. Ownership is `OwnerRef`, a typed `SourceRef` owned by the host application. Ephemeral artifacts and `ArtifactPersistenceSkip` are omitted from checkpoints; `ArtifactPersistenceStore` forces checkpoint persistence. For artifacts, `PolicyReplaceByOrigin` replaces stale artifacts with the same kind/type plus owner/source refs even when the new artifact uses a different `ID`.
371+
372+
Use typed artifact codecs when the host needs structured values to round-trip without manually packing domain data into a raw payload container:
373+
374+
```go
375+
type Fact struct {
376+
Title string `json:"title"`
377+
Body string `json:"body"`
378+
}
379+
380+
desc := contexty.ArtifactCodecDescriptor[Fact]{
381+
TypeID: "example.fact",
382+
Kind: contexty.ArtifactKindRetrievalDocument,
383+
Lifecycle: contexty.ArtifactLifecyclePersistent,
384+
SourceRefs: []contexty.SourceRef{{Namespace: "kb", Kind: "document", ID: "doc-1"}},
385+
MergePolicy: contexty.PolicyReplaceByOrigin,
386+
Budget: &contexty.ArtifactBudgetPolicy{Group: "retrieval", TokenLimit: 2000},
387+
Persistence: contexty.ArtifactPersistenceStore,
388+
Render: func(v Fact) string { return v.Title + ": " + v.Body },
389+
}
390+
artifact, _ := contexty.NewTypedArtifact("fact-1", desc, Fact{
391+
Title: "Boundary",
392+
Body: "Artifacts carry lifecycle and typed source data.",
393+
})
394+
decoded, _ := contexty.DecodeTypedArtifact[Fact](artifact, desc)
395+
_ = decoded
396+
```
339397

340398
Use deltas for immutable state transitions:
341399

0 commit comments

Comments
 (0)