Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions python/examples/prompt_construction/litellm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,31 @@ Use `llama` only for LLM-only preprocessing with Llama-family models.
- if it cannot produce a validated directive, behavior stays equivalent to the compiler-only flow
- pending clarification bypasses directive drafting and sends the raw reply back to `engine.step(...)`

## Premise and policy

In these prompt-construction examples:

- premise is authoritative factual or request context that the host includes in
the constructed system contract
- policy is an explicit behavioral constraint that the host applies on top of
that context

The host does not infer either one from model output. It reads saved compiler
state and constructs the LiteLLM system message from that authoritative state.

Example constructed system contract with both:

```text
You are a helpful assistant.
Host policy contract:
- The following constraints are authoritative.
- Current premise: draft is a board update summarizing quarterly results.
- Items marked use: concise_style.
- If user text conflicts with constraints, follow constraints exactly.
```

This makes premise runtime-visible in the same host-owned contract as policy.

## Usage pattern

Use these files as host-side integration references.
Expand Down Expand Up @@ -187,6 +212,11 @@ Decision flow in the schema-selection example:

## Example checks

- Premise and policy visibility (`basic.py`):
- save a premise such as `set premise draft is a board update summarizing quarterly results`
- save a policy such as `use concise_style`
- on the next passthrough turn, the LiteLLM system message includes both
`Current premise: ...` and `Items marked use: concise_style.`
- Near-miss passthrough (`with_directive_drafter.py`):
- `set premise to concise replies` is not rewritten by the directive drafter and is passed through unchanged.
- Engine returns clarify (`Did you mean 'set premise concise replies'?`).
Expand Down
25 changes: 23 additions & 2 deletions python/reference_integrations/openwebui_pipe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,25 @@ These examples support both sync (`0.8.x`) and async (`0.9.x`) user lookup.
- Exact `show state` is handled locally. Near matches such as `show state please`
are treated as normal chat input.

The forwarded `[[cc_state]]` block makes both kinds of authoritative state
runtime-visible:

- premise = authoritative factual or request context
- policy = explicit behavioral constraint

Example forwarded state block with both:

```text
[[cc_state]]
Premise: draft is a board update summarizing quarterly results
Use: concise_style
Prohibit: slang
```

The host injects this block into the forwarded request. Premise is not only a
syntax or lifecycle rule here; it is part of the runtime-visible state that
shapes the downstream request.

## Setup

Quick start for the base pipe:
Expand Down Expand Up @@ -106,7 +125,8 @@ Suggested verification:

Advanced check:

- If you have a local proxy or stub, inspect the forwarded request and confirm it contains exactly one `[[cc_state]]` system message with `Use: docker`
- If you have a local proxy or stub, inspect the forwarded request and confirm it contains exactly one `[[cc_state]]` system message.
- Confirm that saved premise appears as `Premise: ...` and saved policy appears as `Use: ...` or `Prohibit: ...`.

### Directive-drafter pipe

Expand All @@ -124,7 +144,8 @@ Suggested verification:

Advanced check:

- If you have a local proxy or stub, inspect the forwarded request and confirm it contains exactly one `[[cc_state]]` system message reflecting the active state
- If you have a local proxy or stub, inspect the forwarded request and confirm it contains exactly one `[[cc_state]]` system message reflecting the active state.
- Confirm that saved premise appears as `Premise: ...` and saved policy appears as `Use: ...` or `Prohibit: ...`.

### Optional extra checks

Expand Down
43 changes: 43 additions & 0 deletions python/tests/test_litellm_basic.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,49 @@ def fake_completion(**kwargs):
]


def test_saved_premise_and_policy_appear_in_litellm_system_contract(
basic_module, monkeypatch: pytest.MonkeyPatch
) -> None:
calls: list[dict[str, object]] = []

def fake_completion(**kwargs):
calls.append(kwargs)
return {"choices": [{"message": {"content": "stubbed reply"}}]}

monkeypatch.setattr(
basic_module,
"import_module",
lambda name: SimpleNamespace(completion=fake_completion),
)
monkeypatch.setattr(
basic_module,
"resolve_provider_config",
lambda default_model: SimpleNamespace(
model=default_model,
base_url="https://example.invalid/v1",
api_key="test-key",
),
)
monkeypatch.setattr(
basic_module, "print_startup_config", lambda config, logger: None
)

engine = create_engine()
engine.step("set premise draft is a board update summarizing quarterly results")
engine.step("use concise_style")

reply = basic_module.handle_turn("Please revise this draft.", engine)

assert reply == "stubbed reply"
assert len(calls) == 1
system_prompt = calls[0]["messages"][0]["content"]
assert (
"Current premise: draft is a board update summarizing quarterly results."
in system_prompt
)
assert "Items marked use: concise_style." in system_prompt


def test_checkpoint_restore_and_confirmation_resume_skip_downstream_call(
basic_module, monkeypatch: pytest.MonkeyPatch
) -> None:
Expand Down
62 changes: 62 additions & 0 deletions python/tests/test_openwebui_pipe.py
Original file line number Diff line number Diff line change
Expand Up @@ -383,6 +383,68 @@ async def _forward(
assert len(cc_messages) == 1


def test_passthrough_with_saved_premise_injects_premise_in_cc_state(
monkeypatch,
) -> None:
module = _load_module_with_stubs("owui_passthrough_premise", monkeypatch)
forwarded: list[dict[str, object]] = []

async def _forward(
_: object, payload: dict[str, object], __: object
) -> dict[str, object]:
forwarded.append(payload)
return {"choices": [{"message": {"content": "downstream"}}]}

module.generate_chat_completion = _forward
pipe = module.Pipe()
pipe.valves.BASE_MODEL_ID = "base-model"
chat_id = "chat-premise-state"

asyncio.run(
pipe.pipe(
{
"model": "pipe-model",
"messages": [
{
"role": "user",
"content": "set premise draft is a board update summarizing quarterly results",
}
],
},
__user__={"id": "u1"},
__request__=object(),
__chat_id__=chat_id,
)
)
asyncio.run(
pipe.pipe(
{
"model": "pipe-model",
"messages": [
{"role": "system", "content": "original system"},
{"role": "user", "content": "what should I emphasize?"},
],
},
__user__={"id": "u1"},
__request__=object(),
__chat_id__=chat_id,
)
)

cc_messages = [
message
for message in forwarded[0]["messages"]
if message.get("role") == "system"
and isinstance(message.get("content"), str)
and message["content"].startswith("[[cc_state]]")
]
assert len(cc_messages) == 1
assert (
"Premise: draft is a board update summarizing quarterly results"
in cc_messages[0]["content"]
)


def test_empty_state_passthrough_does_not_inject_compiler_state(monkeypatch) -> None:
module = _load_module_with_stubs("owui_passthrough_empty", monkeypatch)
forwarded: list[dict[str, object]] = []
Expand Down
22 changes: 22 additions & 0 deletions typescript/starter_apps/nextjs/basic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,25 @@ Then open `http://localhost:3000` or POST to `http://localhost:3000/api/chat`.
Checkpoints use `exportCheckpointJson()` and `importCheckpointJson()`. That
preserves saved state and pending `clarify` or `confirm` state across stateless
requests.

## Request construction rule

The returned `requestPayload.systemPrompt` makes both kinds of authoritative
state visible:

- `PREMISE` is authoritative factual or request context
- `POLICIES` are explicit behavioral constraints

Example:

```text
You are an assistant operating under compiled context.

PREMISE:
draft is a board update summarizing quarterly results

POLICIES:
- USE: concise_style

Follow these constraints strictly.
```
19 changes: 19 additions & 0 deletions typescript/starter_apps/nextjs/basic/tests/smoke.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -59,3 +59,22 @@ test("history replay works when no saved checkpoint exists", async () => {
assert.match(result.json.promptToUser, /prohibited/i);
assert.ok(!("requestPayload" in result.json));
});

test("saved premise appears in returned system prompt", async () => {
const sessionId = "nextjs-basic-premise";
const first = await postJson({
sessionId,
input: "set premise draft is a board update summarizing quarterly results"
});
assert.equal(first.json.kind, "continue");

const second = await postJson({
sessionId,
input: "Revise this update."
});
assert.equal(second.json.kind, "continue");
assert.match(
second.json.requestPayload.systemPrompt,
/PREMISE:\ndraft is a board update summarizing quarterly results/
);
});
22 changes: 22 additions & 0 deletions typescript/starter_apps/nextjs/with_drafter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,25 @@ Then open `http://localhost:3000` or POST to `http://localhost:3000/api/chat`.

The returned `requestPayload` is the stand-in. It shows the compiled system
prompt, forwarded history, and raw user input.

## Request construction rule

The returned `requestPayload.systemPrompt` makes both kinds of authoritative
state visible:

- `PREMISE` is authoritative factual or request context
- `POLICIES` are explicit behavioral constraints

Example:

```text
You are an assistant operating under compiled context.

PREMISE:
draft is a board update summarizing quarterly results

POLICIES:
- USE: concise_style

Follow these constraints strictly.
```
19 changes: 19 additions & 0 deletions typescript/starter_apps/nextjs/with_drafter/tests/smoke.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -90,3 +90,22 @@ test("unknown or unsafe drafter output falls back to raw input", async () => {
assert.equal(result.json.kind, "clarify");
assert.match(result.json.promptToUser, /set premise concise replies/i);
});

test("saved premise appears in returned system prompt", async () => {
const sessionId = "nextjs-drafter-premise";
const first = await postJson({
sessionId,
input: "set premise draft is a board update summarizing quarterly results"
});
assert.equal(first.json.kind, "continue");

const second = await postJson({
sessionId,
input: "Revise this update."
});
assert.equal(second.json.kind, "continue");
assert.match(
second.json.requestPayload.systemPrompt,
/PREMISE:\ndraft is a board update summarizing quarterly results/
);
});