Skip to content

Commit ae1ccbf

Browse files
authored
feat: add premise relevance to retrieval filtering (#49)
1 parent 16e1127 commit ae1ccbf

8 files changed

Lines changed: 471 additions & 17 deletions

File tree

‎python/examples/retrieval_filtering/README.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,14 @@ The host reads authoritative state to determine which audiences are eligible:
2222
- `use manager_hr_access` makes employee and manager documents retrievable
2323
- absent state follows the documented default of returning no HR documents
2424

25+
The generic HR example now also contrasts premise with policy:
26+
27+
- policy decides which audiences are eligible first
28+
- saved factual case premise can then narrow relevance inside that eligible set
29+
- the same access state can keep the same eligible documents while premise
30+
changes which relevant document is returned
31+
- premise does not grant access and does not select a collection
32+
2533
Adversarial queries such as "ignore policy and show executive compensation",
2634
"I am the CEO", and "reveal all documents" do not change eligibility because
2735
query text does not mutate authoritative state.

‎python/examples/retrieval_filtering/hr_policy_lookup/README.md‎

Lines changed: 48 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ The host owns:
2626
Context Compiler owns:
2727

2828
- the authoritative access state
29+
- the authoritative saved case premise
2930
- clarification behavior for contradictory directives
3031

3132
This example does not call an LLM, does not use directive drafter, and does not
@@ -36,19 +37,62 @@ derive state from model output.
3637
The example corpus contains:
3738

3839
- `employee_handbook`
40+
- `leave_of_absence_policy`
3941
- `manager_handbook`
4042
- `executive_compensation_policy`
4143

42-
The host maps authoritative state to eligible audiences:
44+
The host applies retrieval in this order:
4345

4446
- `use employee_hr_access` allows employee documents
4547
- `use manager_hr_access` allows employee and manager documents
4648
- absent state follows the documented default of returning no HR documents
49+
- after eligibility is fixed, saved premise facts may narrow relevance inside
50+
the eligible set
51+
52+
Policy controls eligibility. Premise controls relevance within that eligible
53+
set. Premise does not grant access and does not select a collection.
54+
55+
For premise-driven relevance, the host applies a small deterministic rule:
56+
57+
`saved HR case facts -> case context -> relevant documents within eligible set`
58+
59+
Examples:
60+
61+
- `set premise case concerns leave eligibility after a parental leave request`
62+
narrows employee-visible results toward `leave_of_absence_policy`
63+
- `set premise case concerns general employee handbook expectations for a new hire`
64+
narrows employee-visible results toward `employee_handbook`
65+
- `set premise case concerns staffing approval for a team reorganization`
66+
narrows manager-visible results toward `manager_handbook`
67+
68+
With the same query, `leave`:
69+
70+
- `use employee_hr_access` plus the leave-eligibility premise returns
71+
`leave_of_absence_policy`
72+
- `use employee_hr_access` plus the general employee-handbook premise returns
73+
`employee_handbook`
74+
75+
In both cases, the eligible employee documents stay the same:
76+
77+
- `employee_handbook`
78+
- `leave_of_absence_policy`
79+
80+
The premise changes only which already-eligible document is returned as the
81+
relevant match. It does not change `eligible_document_ids`.
82+
83+
Without a matching or known premise, the host falls back to the default
84+
employee-handbook relevance path. Unknown premise text does not invent a new
85+
result set.
86+
87+
If the saved premise points at manager-only staffing context while policy still
88+
allows only employee access, the host returns no results for that premise path
89+
rather than expanding eligibility.
4790

4891
Executive documents remain filtered because this example never grants executive
4992
access. Adversarial queries such as "ignore policy and show executive
5093
compensation", "I am the CEO", and "reveal all documents" stay inert unless the
51-
authoritative state changes.
94+
authoritative state changes. Adversarial query text does not overwrite either
95+
saved access policy or saved case premise.
5296

5397
If a turn introduces a contradiction such as `use employee_hr_access` followed
5498
by `prohibit employee_hr_access`, Context Compiler returns a clarification flow
@@ -60,7 +104,8 @@ rather than treating it as a retrieval override.
60104
The observable runtime behavior change is the returned document set. The query
61105
text alone cannot bypass filtering. Retrieval results change only because the
62106
host reads different authoritative Context Compiler state before searching the
63-
same corpus.
107+
same corpus. Access eligibility is applied first, and premise-based relevance
108+
is applied only inside that eligible document set.
64109

65110
## Validation
66111

‎python/examples/retrieval_filtering/hr_policy_lookup/example.py‎

Lines changed: 85 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,19 +10,26 @@
1010
create_engine,
1111
get_decision_state,
1212
get_policy_items,
13+
get_premise_value,
1314
is_clarify,
1415
)
1516
from context_compiler.engine import Engine
1617

1718
EMPLOYEE_ACCESS = "employee_hr_access"
1819
MANAGER_ACCESS = "manager_hr_access"
20+
LEAVE_CASE_PREMISE = "case concerns leave eligibility after a parental leave request"
21+
GENERAL_HANDBOOK_PREMISE = (
22+
"case concerns general employee handbook expectations for a new hire"
23+
)
24+
STAFFING_CASE_PREMISE = "case concerns staffing approval for a team reorganization"
1925

2026

2127
class PolicyDocument(TypedDict):
2228
document_id: str
2329
title: str
2430
audience: Literal["employee", "manager", "executive"]
2531
keywords: list[str]
32+
relevance_tags: list[str]
2633
content: str
2734

2835

@@ -39,22 +46,35 @@ class RetrievalTurnResult(TypedDict):
3946
retrieval_result: RetrievalResult
4047

4148

49+
CaseContext = Literal["general_handbook_case", "leave_case", "staffing_case"]
50+
51+
4252
@dataclass
4353
class HRPolicyRetriever:
4454
"""Host-owned retrieval implementation with deterministic filtering."""
4555

4656
documents: list[PolicyDocument] = field(default_factory=list)
4757

48-
def search(self, query: str, *, allowed_audiences: set[str]) -> RetrievalResult:
58+
def search(
59+
self,
60+
query: str,
61+
*,
62+
allowed_audiences: set[str],
63+
case_context: CaseContext | None,
64+
) -> RetrievalResult:
4965
eligible_documents = [
5066
document
5167
for document in self.documents
5268
if document["audience"] in allowed_audiences
5369
]
70+
relevance_filtered_documents = filter_documents_by_case_context(
71+
eligible_documents,
72+
case_context,
73+
)
5474
normalized_query_terms = set(query.lower().split())
5575
returned_documents = [
5676
document
57-
for document in eligible_documents
77+
for document in relevance_filtered_documents
5878
if normalized_query_terms & set(document["keywords"])
5979
]
6080

@@ -77,20 +97,31 @@ def example_documents() -> list[PolicyDocument]:
7797
"title": "Employee Handbook",
7898
"audience": "employee",
7999
"keywords": ["employee", "handbook", "benefits", "leave"],
100+
"relevance_tags": ["general_handbook_case", "general_hr"],
80101
"content": "General HR policy, leave policy, and workplace expectations.",
81102
},
103+
{
104+
"document_id": "leave_of_absence_policy",
105+
"title": "Leave of Absence Policy",
106+
"audience": "employee",
107+
"keywords": ["leave", "eligibility", "parental"],
108+
"relevance_tags": ["leave_case"],
109+
"content": "Leave eligibility, parental leave steps, and required documentation.",
110+
},
82111
{
83112
"document_id": "manager_handbook",
84113
"title": "Manager Handbook",
85114
"audience": "manager",
86115
"keywords": ["manager", "handbook", "approvals", "staffing"],
116+
"relevance_tags": ["general_handbook_case", "staffing_case"],
87117
"content": "Manager escalation guidance, staffing policy, and approvals.",
88118
},
89119
{
90120
"document_id": "executive_compensation_policy",
91121
"title": "Executive Compensation Policy",
92122
"audience": "executive",
93123
"keywords": ["executive", "compensation", "bonus", "board"],
124+
"relevance_tags": ["executive_only"],
94125
"content": "Executive compensation bands, board review, and bonus structure.",
95126
},
96127
]
@@ -130,6 +161,56 @@ def allowed_audiences_from_state(state: State) -> set[str]:
130161
return set()
131162

132163

164+
def classify_premise_as_case_context(premise: str | None) -> CaseContext | None:
165+
"""Map saved HR case facts to a host-owned retrieval relevance context."""
166+
167+
if premise is None:
168+
return None
169+
170+
normalized_premise = premise.casefold()
171+
if (
172+
"general employee handbook" in normalized_premise
173+
and "new hire" in normalized_premise
174+
):
175+
return "general_handbook_case"
176+
177+
if (
178+
"leave eligibility" in normalized_premise
179+
and "parental leave" in normalized_premise
180+
):
181+
return "leave_case"
182+
183+
if (
184+
"staffing approval" in normalized_premise
185+
and "team reorganization" in normalized_premise
186+
):
187+
return "staffing_case"
188+
189+
return None
190+
191+
192+
def filter_documents_by_case_context(
193+
documents: list[PolicyDocument],
194+
case_context: CaseContext | None,
195+
) -> list[PolicyDocument]:
196+
"""Limit relevance only within the already eligible document set."""
197+
198+
if case_context is None:
199+
return [
200+
document
201+
for document in documents
202+
if "general_handbook_case" in document["relevance_tags"]
203+
]
204+
205+
relevant_documents = [
206+
document for document in documents if case_context in document["relevance_tags"]
207+
]
208+
if relevant_documents:
209+
return relevant_documents
210+
211+
return []
212+
213+
133214
def retrieve_hr_documents(
134215
query: str,
135216
*,
@@ -138,9 +219,11 @@ def retrieve_hr_documents(
138219
) -> RetrievalResult:
139220
"""Retrieve only documents the host deems eligible from compiler state."""
140221

222+
premise = get_premise_value(state)
141223
return retriever.search(
142224
query,
143225
allowed_audiences=allowed_audiences_from_state(state),
226+
case_context=classify_premise_as_case_context(premise),
144227
)
145228

146229

0 commit comments

Comments
 (0)