Status: Ready for implementation
Audience: 负责 Agent、MetaChain、skills、Runtime、评测和发布的维护者
Scope: 把现有“阶段级静态工具列表 + Agent 内直接执行”迁移为可回放的 Tool Interaction、按需工具暴露、Runtime-governed Effect、完整评测与逐阶段发布
Owner: AI-Researcher maintainers
Last updated: 2026-07-31
Governing design: Context-Aware Tool Use and Governed Tool Effects
Related governing designs: Durable Research Runtime and Stage Continuation, Experience-Driven Research Loop, and Verified Research Memory
本计划完成时,AI-Researcher 必须具备以下生产能力,而不是只有一个离线 tool-search demo:
- 每个 model turn 都能显式产生
DIRECT | CLARIFY | DISCOVER | EXECUTE之一;COMPLETE | HANDOFF是独立控制结果,不是外部工具。 - 每次决策先做 stage/auth/config/environment/data-flow/risk/budget 硬过滤, 再做相关性排序;retriever 永远不能授权执行。
- 小工具集在 schema 预算内直接暴露;只有超过预算,或公开 SelectionPolicy 中 冻结的 development/baseline narrowing override 命中时才 progressive discovery; evaluator-only gold labels 永远不进入 decision plane。
- 工具集合随已提交 ToolFeedback、失败、证据缺口、subgoal 和 availability 变化重新选择。
- 所有 model/tool 调用最终都成为现有 Long-Task Runtime 所有的 Effect;每次 dispatch 有独立 Physical Invocation、Budget Envelope reservation、receipt、 reconciliation 和 Durable Transition。
- 工具返回是 typed、bounded、artifact-backed
ToolFeedback;不再依赖字符串Result判断成功,也不与 Experiment Attempt 的 Observation 混名。 - 每个决策、候选、暴露集合、model proposal、授权结果、工具尝试、失败和 feedback 都可重放,并进入正确的 cache/provenance identity。
- static、hard-filter、dense、hybrid、trajectory 五个 arm 能在同一个工具使用 Evaluation Contract、模型和预算下比较。
- 通过 shadow、只读 canary、危险工具 canary 和 durable rollout 后,删除生产 直接 callable dispatch、control-as-tool、全局覆盖式注册、重复 cache/trace authority 与 MD5 检索 fallback。
文档、接口类或通过若干 loader unit tests 不等于交付。只有本计划的 Definition of Done 全部有当前代码、测试、运行 artifact 和 acceptance manifest 证据时, 才能把 governing design 标记为 Accepted。
Phase 0 在任何行为修改前生成 content-addressed baseline manifest,至少记录:
- git revision、依赖 lock/fingerprint、Python、OS 和 tokenizer 版本;
- 两个 entry flow 的 Agent/role/stage 列表;
- 每个 Agent 的 callable 名称、顺序、完整 provider schema 和 token 数;
tool_choice、SiliconFlow 等 provider-specific effective choice、parallel flag;- 完整 model request、model response、tool proposal、call args、返回、异常、handoff、 context patch 和 stage artifact digests;
- model/tool call 数、input/schema/output tokens、p50/p95 latency 和 cost;
- cache key、hit/miss 和 cached transcript digest;
- 至少一组 provided-idea 和一组 reference-ideation fixture;
- 当前 45 个 decorator tool、SKILL manifest tool 和 Agent-bound wrapper 的清单与 重叠/缺失报告。
当前调研测得 Agent 每轮约 264--1,637 schema tokens,只作为计划依据。发布比较 必须使用 Phase 0 自动生成的 raw measurement,而不是复制文档数字。
工具选择不等待整个 Durable Runtime 完成,但执行声明有严格前置:
| Tool-use phase | 可并行开始 | Durable Runtime 前置 | 允许的声明 |
|---|---|---|---|
| T0 trace baseline | 立即 | 无 | “现有调用可观测” |
| T1 canonical catalog | 立即 | 无 | “工具身份/schema 可校验” |
| T2 pure ToolInteraction | 立即 | 无 | “决策可确定性测试” |
| T3 shadow integration | 立即 | 无 | “反事实选择对 active path 无影响” |
| T4 read-only enforcement | T0--T3 后 | Runtime Phase 2 的 effect journal、reservation、fencing、receipt/reconcile | “按需暴露已启用;每次 read-only call 也是 governed Effect” |
| T5 hybrid/trajectory selection | T4 后 | 同 T4;committed ToolFeedback 必须来自真实 tool Effect 或 tool preparation rejection | “动态重选已启用” |
| T6 execution/output convergence | T5 后 | Runtime Phase 2 + execution Adapter contract suites | “全部工具输出 typed,legacy output bridge 归零” |
| T7 all-stage migration | Runtime Phase 5/6 配合 | native stage Adapters 与 entry migration | “两个 flow 共用目标路径” |
| T8 acceptance/deletion | Runtime release gates 配合 | authoritative journal、rollout/rollback | 完整生产声明 |
T3 以前的 LegacyMetaChainToolAdapter 只用于 legacy/shadow active lane,仍是带 typed
trace 的 direct-call migration bridge,任何发布说明都不得称其 durable。进入 T4 后,
active adaptive profile 禁止 direct callable:旧 binding 只能作为真实 Runtime Effect
后的 LegacyToolExecutionAdapter,由 Runtime 产生 Effect/Physical Invocation/fence/
receipt/settlement;Adapter 可暂时把输出标为 LEGACY_UNTYPED,但不拥有执行或重试
语义。不得合成伪 Effect ID。
T3 之前必须完成:
- 所有内部工具尝试都进入 trace,包括 unknown tool、JSON parse、schema、异常、 denied 和 unavailable;
ToolModule不再无条件记录success=True;AgentModulecache identity 绑定真实 active tool configuration;- trace 明确区分 control action、model Effect、tool Effect 和 flow-level helper;
- 日志/trace 写入失败不能静默伪造一次成功调用;
- provider request schema token 由实际序列化内容自动计算。
T4 之前必须完成:
- 每个 active Agent 静态列表都能编译为 canonical catalog + static policy;
- 同名不同 schema/binding 冲突全部显式解决;
- adaptive provider profile 通过 optional tool-use 和 canonical control tests;
DIRECT/CLARIFY有 stage completion/wait 语义,不能依赖“没有 tool call 就 break”。
T4 active enforcement 之前必须完成 Durable Runtime plan 的 effect journal、Physical Invocation、 reservation、fencing、retry ownership、artifact commit 与 Adapter contract。
基础 ToolInteraction 不依赖 memory 上线:decision_memory 是 closed optional
input,无 recall 时写显式 no-recall identity。T0--T5 的 memory case 可用
deterministic fixture 完成,不得因此声称生产 Recall Decision Outcome 已交付。
普通 memory-off/non-memory stage 使用 no-decision-memory;registered control arm
仍传 preallocated Decision Intent + source_status=NOT_REQUESTED、null Recall
Input/Context、zero-card/zero-token binding,并写 terminal outcome,不能借 None
逃出 denominator,也不能误记为 blocked。
生产启用 memory-informed tool decision 之前,Verified Research Memory plan 至少要 完成对应实现阶段(当前计划 Phase 3)的 typed Decision Intent、Recall Input Snapshot、Recall Context、per-card disposition 和 durable Recall Decision Outcome Store contracts。集成还必须证明:
- Tool Decision Record 先不可变提交;模型 disposition 只是 untrusted draft;Runtime
绑定实际 decision/authorization 后提交 Recall Decision Outcome,再由幂等
MemoryToolDecisionLinkedevent 关联; - memory-informed
EXECUTE的 guardedDISPATCHED必须等待 outcome link; crash/replay 不产生孤儿、重复逻辑 outcome 或未记录的外部行动; - empty/degraded recall 不扩大候选或触发 global fallback;
- Evidence Card 永远不能改变 hard eligibility、approval 或 Runtime authorization;
- outcome evaluator 使用 opaque trial identity,评分后由独立 manipulation audit 连接 Tool Decision/Recall Decision Outcome,不把 actor memory 暴露给 evaluator。
- 每个 memory-governed model Effect 预分配新的 logical decision slot/Decision Intent;
DIRECT/CLARIFY/DISCOVER/EXECUTE/REJECTED各自关闭当前 Intent。discovery 后、 clarification resume 后和 committed ToolFeedback 后的 model Effect 使用新的 child Intent,并可绑定 parent outcome digest;不能复用旧 Intent。 - 下一 model request 从 committed artifacts 重建,删除前一 turn rendered Recall
Context bytes,只携带 bounded trajectory facts 和 parent outcome digest;新 turn
重新 recall,或用注册的
source_status=not_requestedzero-card/null-context control outcome。
若 memory Phase 3 未就绪,rollout 继续支持 no-recall ToolInteraction;不得保留一条 没有 terminal Recall Decision Outcome 的临时 production recall 通道。
Agent stage policy data
↓
inno.tool_use.ToolInteraction (pure)
↓ directives only
runtime.adapters.tool_interaction
↓ EffectSpec
LongTaskRuntime private control/executor protocol
↓ authorized Physical Invocation
runtime ToolExecutionAdapter
↓ raw evidence
LongTaskRuntime receipt selection/settlement
↓ committed ToolFeedback input
inno.tool_use.ToolInteraction
旁路但不混入 per-turn Interface:Run admission/Snapshot builder 调用独立
CapabilityCatalogCompiler,把 legacy/skill fragments 编译为 pinned catalog;Runtime
Adapter 验证 Selection Bundle artifacts 并构造 read-only ToolInteraction
Implementation。二者都在 advance() 前完成。
禁止依赖:
inno.tool_use不得 importlitellm, Docker socket, Browser environment, mutable Registry, SQLite store 或 concrete tool Implementation;- Agent factory 不得在目标状态 import concrete tools 来决定暴露集合;
- retriever/ranker 不得 import ToolExecutionAdapter;
- ToolExecutionAdapter 不得 import selection/ranking policy;
- CLI/Web 不得 import ToolInteraction、Runtime private authorization、worker、store 或 execution Adapter;
- evaluator 不得通过 ToolInteraction/Agent 写 Verification Record;
- actor/decision package 不得 import evaluator-label schema 或 private corpus namespace;
flowcache.py不得成为 external Effect execution 或 retry authority。
新增 dependency test 应解析 import graph,并在 CI 中拒绝上述反向边。
research_agent/inno/tool_use/
__init__.py
interaction.py
canonical.py
catalog.py
policy.py
data_flow.py
bundle.py
views.py
selection.py
feedback.py
protocol.py
tracing.py
adapters/
__init__.py
catalog_legacy.py
catalog_skill.py
provider_native.py
provider_prompt.py
research_agent/runtime/
_tool_authorization.py
adapters/
tool_interaction.py
tool_execution.py
tool_shadow.py
research_agent/inno/evals/
tool_use.py
benchmark/tool_use/
__init__.py
schemas.py
contracts/tool-use-v1.yaml
contracts/release-criteria-v1.yaml
cases/actor/
cases/evaluator_private/
fixtures/
run.py
analyze.py
validate_manifest.py
tests/test_tool_use/
test_interaction_contract.py
test_catalog_contract.py
test_policy.py
test_data_flow.py
test_selection_bundle.py
test_views.py
test_selection.py
test_feedback.py
test_protocol_contract.py
test_shadow_inertness.py
test_meta_chain_integration.py
test_cache_identity.py
test_dependency_rules.py
test_evaluator_label_isolation.py
test_trace_causality.py
test_acceptance_manifest.py
test_rollout_rollback.py
tests/test_runtime/
test_tool_authorization.py
test_tool_effect_adapters.py
test_tool_effect_faults.py
类型放在拥有该 Seam 的文件中,不新建 catch-all models.py、interfaces.py 或
utils.py。
| File/Module | 实施动作 |
|---|---|
research_agent/inno/core.py |
T0 完整 trace;T3 接 migration Adapter;T4 从 Prepared model Effect 构造 request;T6 删除 direct callable execution |
research_agent/inno/types.py |
增加迁移期 policy/profile refs;最终收窄 Agent.functions 与 string Result 职责 |
research_agent/inno/registry.py |
T1 只作为 legacy binding source;禁止覆盖式 identity resolution;T8 删除 metadata authority |
research_agent/inno/tools/__init__.py |
T1 停止为 catalog 做 eager import;T8 删除 recursive production import |
research_agent/inno/skills/base.py |
manifest 指向 canonical descriptor/schema digest,不维持独立可漂移 schema |
research_agent/inno/skills/loader.py |
保留 manifest-only scan,输出 catalog fragment,实际校验 required config metadata |
research_agent/inno/skills/registry.py |
停止把 skill tool 注入全局 Registry;迁移为 catalog source,最终删除旧 search Interface |
research_agent/inno/skills/search.py |
删除 MD5 fallback;可复用逻辑迁入 private selection Implementation;T8 删除浅 Interface |
research_agent/inno/workflow/flowcache.py |
trace/cache identity 完整化;停止独立工具执行和结果 cache authority |
research_agent/inno/workflow/cache_identity.py |
增加 catalog/policy/protocol/toolset/trajectory/Adapter digests |
research_agent/inno/evals/trace.py |
closed typed tool-decision/execution projection;不再遗漏 internal call |
research_agent/inno/evals/metrics.py |
加 tool-use diagnostic metrics;end-state Verification 仍是 primary |
research_agent/runtime/context.py |
Snapshot 绑定 catalog/policy/protocol/Adapter bundle digests;构造 actor-safe scope 与 Runtime-only authorization envelope,禁止 transitive private identity 泄漏 |
research_agent/run_infer_plan.py |
两个 flow 逐阶段改用 policy ref 与共同 Adapter |
research_agent/run_infer_idea.py |
同上 |
research_agent/inno/agents/inno_agent/*.py |
callable selection 迁移为 stage capability/completion policy;移除 control-as-tool |
| Component | 唯一责任 | 禁止 |
|---|---|---|
tool_use/tracing.py |
canonical Tool Decision/event payload schemas 和 canonical digest | 写 journal、拼最终评测 report |
| Runtime journal/store | append-only mutation authority、顺序、linkage | 改写 Tool Decision Record、计算研究结论 |
inno/evals/trace.py |
journal -> ResearchRunTrace 只读 projection Adapter |
成为第二事实源 |
inno/evals/tool_use.py / metrics.py |
从 frozen trace + label artifact 纯计算 metrics | 读取 actor mutable state、写 Verification |
benchmark/tool_use/run.py |
装载 manifest、分配 Run、保存 raw artifacts | 内嵌另一套 metric 公式 |
benchmark/tool_use/analyze.py |
调用 pinned metric Implementation、CI 和 gate evaluator | 接受临时阈值覆盖 frozen criteria |
所有持久化模型使用 Pydantic v2、extra="forbid"、frozen=True、显式
schema_version、closed enum 与 canonical JSON。时间用 UTC;digest 使用 SHA-256。
未知 enum/schema version 必须 fail closed。
frozen=True 不是深不可变。所有 JSON schema、canonical args、policy map 和 provider
projection 在进入 domain model 时必须转换为递归 immutable CanonicalJsonValue
(object key 排序、array 保序、拒绝 NaN/Infinity/duplicate key),digest 计算自同一组
canonical bytes。Provider Adapter 只能从该值生成一次性 wire dict,不能把可变 dict
保存回 record。
所有 content digest 使用同一个 canonical.py Implementation:覆盖 schema version
和全部语义字段,排除自身 *_digest、artifact 存储位置、签名包装与非语义写入时间;
ref/digest pair 必须在使用前验证。禁止每个 Module 自己拼 JSON 或把 digest 字段递归
包含进自身。
catalog.py 至少定义:
class CapabilityDescriptor(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
schema_version: Literal["1"]
capability_id: str # namespace/name@version
source_kind: Literal["legacy_registry", "skill_manifest", "builtin"]
source_ref: str
source_digest: str
provider_name_hint: str
summary: str
action: str
object: str
when_to_use: tuple[str, ...]
when_not_to_use: tuple[str, ...]
positive_examples: tuple[str, ...]
negative_examples: tuple[str, ...]
confused_with: tuple[str, ...]
input_schema: CanonicalJsonObject
output_schema: CanonicalJsonObject
input_schema_digest: str
output_schema_digest: str
descriptor_digest: str
stage_tags: tuple[str, ...]
required_config: tuple[str, ...]
required_auth_scopes: tuple[str, ...]
execution_binding: ExecutionBinding
effect_policy: CapabilityEffectPolicy
data_policy: CapabilityDataPolicy
output_policy: CapabilityOutputPolicy
dependency_policy: CapabilityDependencyPolicy
cost_hint: CapabilityCostHint | Nonedescriptor_digest 覆盖全部行为相关字段,但不包含 live availability、secret、
近期成功率等运行时观测。非权威历史统计可存于 derived ranking feature artifact,
必须有时间窗和来源,不得改变 hard authorization。
至少表达:
class CapabilityEffectPolicy(BaseModel):
schema_version: Literal["1"]
side_effect: Literal["none", "read", "write", "destructive", "external_publish"]
execution_kind: Literal["in_process", "subprocess", "docker", "browser"]
idempotency: Literal[
"pure", "idempotent", "queryable", "reexecutable", "non_reconcilable"
]
checkpoint_policy: Literal[
"RESUMABLE", "RESTARTABLE", "RECONCILE_ONLY", "NON_RESUMABLE"
]
cache_policy: Literal["never", "exact_receipt", "content_addressed_read"]
parallel_safety: Literal["serial", "independent_read", "stateful"]
timeout_seconds: int
worst_case_reservation: WorstCaseUsage
approval_class: str | None
class WorstCaseUsage(BaseModel):
schema_version: Literal["1"]
model_calls: int
tool_calls: int
input_tokens: int
output_tokens: int
wall_time_ms: int
cpu_time_ms: int
gpu_time_ms: int
network_bytes: int
output_bytes: int
cost_microunits: int
class ExecutionBinding(BaseModel):
schema_version: Literal["1"]
operation_id: str
adapter_kind: Literal["in_process", "subprocess", "docker", "browser"]
binding_ref: ArtifactRef
binding_digest: str
adapter_contract_version: str
adapter_contract_digest: str
class CapabilityDataPolicy(BaseModel):
schema_version: Literal["1"]
accepted_input_classes: tuple[str, ...]
output_class: str
allowed_destination_classes: tuple[str, ...]
network_access: Literal["none", "restricted", "open_world"]
evaluator_private_allowed: Literal[False]
policy_digest: str
class CapabilityOutputPolicy(BaseModel):
schema_version: Literal["1"]
allowed_content_types: tuple[str, ...]
max_raw_bytes: int
max_inline_bytes: int
max_model_tokens: int
artifact_required_above_bytes: int
structural_redaction_policy_digest: str
projection_renderer_digest: str
policy_digest: str
class CapabilityDependencyPolicy(BaseModel):
schema_version: Literal["1"]
prerequisite_capability_ids: tuple[str, ...]
successor_capability_ids: tuple[str, ...]
incompatible_capability_ids: tuple[str, ...]
required_evidence_kinds: tuple[str, ...]
policy_digest: str
class CapabilityCostHint(BaseModel):
schema_version: Literal["1"]
measurement_window: str
source_artifact_ref: ArtifactRef
source_artifact_digest: str
sample_count: int
p50_wall_time_ms: int
p95_wall_time_ms: int
p50_cost_microunits: int
class CatalogSourceRef(BaseModel):
schema_version: Literal["1"]
source_kind: Literal["legacy_registry", "skill_manifest", "builtin"]
source_ref: ArtifactRef
source_digest: str
adapter_contract_digest: strModel Effect validator 要求 (model_calls, tool_calls) == (1, 0);Tool Effect request
要求 (0, 1);所有 usage 数值非负,currency 使用整数 microunits,不用 float。
Catalog compilation 验证 effect policy 与 execution Adapter contract 兼容。例如
non_reconcilable 外部写不能声明 RESTARTABLE,stateful Browser 不能声明
independent_read,超出 in-process bounded limit 的工具必须用 subprocess/container。
class CapabilityCatalog(BaseModel):
schema_version: Literal["1"]
catalog_id: str
descriptors: tuple[CapabilityDescriptor, ...] # sorted by capability_id
source_fragments: tuple[CatalogSourceRef, ...]
compiler_version: str
compiler_digest: str
catalog_digest: str编译命令的目标 Interface:
python -m research_agent.inno.tool_use.catalog check
python -m research_agent.inno.tool_use.catalog build --output ARTIFACT_DIR
python -m research_agent.inno.tool_use.catalog diff OLD NEW
check 必须在不调用任何 Tool Implementation 的情况下发现 identity/schema/policy
冲突。需要环境 wrapper 才能生成 executable schema 的 legacy tool,使用显式 fixture
binding;不能通过递归 import 全库并吞掉异常来“检查”。
Policy 是 content-addressed data,由一个 pinned interpreter 解释,不是任意 Python subclass plugin。建议 v1 schema:
schema_version: "1"
policy_id: researcher/adaptive-v1
mode: adaptive # static | shadow | adaptive
selection_bundle_ref: artifact://tool-selection/bundle-v1.json
selection_bundle_digest: "..."
stage_allowlists:
prepare: ["..."]
coding_plan: ["..."]
frozen_protocol_denies:
- side_effect: write
- side_effect: destructive
schema_budget:
absolute_tokens: 2048
max_context_fraction: 0.02
small_set_bypass:
enabled: true
narrowing_overrides:
- override_id: coding/file-vs-web-v1
capability_ids: ["builtin/file.read@1", "builtin/web.search@1"]
approved_development_artifact_ref: artifact://tool-evals/dev-confusion-v1.json
approved_development_artifact_digest: "..."
selection:
strategy: hybrid_trajectory_v1
dense_degradation: lexical_only # lexical_only | expand | clarify | wait | fail
low_confidence: expand_then_clarify
max_discovery_steps: 2
max_model_corrections: 1
reretrieve_on:
- tool_feedback
- typed_failure
- evidence_gap
- subgoal_change
- availability_change
fallback:
scope: original_stage_allowlist # never global registry
only_if_within_schema_budget: true2048 和 0.02 是初始 compatibility profile 值,依据当前最大约 1,637 tokens
留出控制 schema 余量;不是跨模型结论。T0 必须测量各 provider tokenizer,并在
adaptive 结果揭晓前冻结正式 policy。任何调整产生新 policy digest 和新 Research
Run。
SelectionBundle 关闭所有“代码默认值没进 digest”的后门:
class IndexedDescriptorBinding(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
schema_version: Literal["1"]
capability_id: str
descriptor_digest: str
class DenseRuntimeProfile(BaseModel):
model_config = ConfigDict(extra="forbid", frozen=True)
schema_version: Literal["1"]
inference_runtime_version: str
inference_runtime_digest: str
kernel_abi: str
platform_abi: str
cpu_feature_mask: str
thread_count: Literal[1]
device: Literal["cpu"]
vector_dtype: Literal["float32"]
score_encoding: Literal["signed_int64_fixed_point"]
score_scale: int
rounding_mode: Literal["half_even"]
profile_digest: str
class SelectionBundle(BaseModel):
schema_version: Literal["1"]
bundle_id: str
interpreter_version: str
interpreter_digest: str
tokenizer_ref: ArtifactRef
tokenizer_digest: str
schema_estimator_version: str
schema_estimator_digest: str
lexical_analyzer_config: CanonicalJsonObject
lexical_analyzer_digest: str
embedding_model_ref: ArtifactRef | None
embedding_model_revision: str | None
embedding_model_digest: str | None
dense_index_ref: ArtifactRef | None
dense_index_digest: str | None
indexed_catalog_digest: str | None
indexed_descriptors: tuple[IndexedDescriptorBinding, ...]
indexed_corpus_digest: str | None
feature_input_projection_digest: str | None
dense_runtime_profile: DenseRuntimeProfile | None
normalization_digest: str
fusion_config: CanonicalJsonObject
fusion_digest: str
confidence_thresholds: CanonicalJsonObject
renderer_version: str
renderer_digest: str
stable_tie_break: Literal["capability_id"]
bundle_digest: strstatic/lexical-only policy 对 optional dense refs/profile 使用显式 None,对
indexed_descriptors 使用显式空 tuple,不能运行时偷用本地模型。
NarrowingOverride 只能引用 adaptive 结果揭晓前冻结的 development/baseline
artifact,且 schema 禁止 case ID、expected/forbidden、milestone、minefield、answer、
treatment 字段。Run spec、ToolInteraction request、cache、Tool Decision Record 和
acceptance manifest 全部绑定 bundle_digest。
dense-enabled bundle 的 admission validator 必须证明:
indexed_catalog_digest == active catalog_digest;indexed_descriptors按 capability ID 排序,和 Catalog 的(capability_id, descriptor_digest)集合完全相等;indexed_corpus_digest覆盖该 ordered tuple、feature/input projection 和 embedding model identity;- 当前 worker fingerprint 精确匹配
DenseRuntimeProfile; - query/document score 在 threshold/fusion 前按 profile 转为 canonical signed-int64 fixed point。
V1 不允许 partial dense index。陈旧、catalog-swap、descriptor 缺失或 runtime fingerprint 不匹配均产生 typed bundle rejection,并按 pinned policy 显式 lexical-only/wait/fail;不能把未索引 capability 静默丢掉。clean-process、不同 worker 和允许的 CPU 主机上必须有 byte-golden tests;GPU、并行归约和未固定 kernel 禁止进入 active selection。
至少包含:
- stage/role 到 capability class 的 allow/deny;
- principal/auth scope 与 required config 规则;
- source/destination data classification 和 egress matrix;
- destructive/external-publish approval requirement;
- per-capability argument constraints;
- allowed execution Adapter bundle;
- idempotency/reconcile/cache requirements;
- blocked-diagnostic identity/summary/reason/remediation visibility rules;
- ToolFeedback inline/artifact/token/redaction policy;
- maximum tool calls, tool result tokens, wall time and cost within the enclosing Budget Envelope;
- availability attestation TTL 与 stale behavior。
SelectionPolicy 可以缩小候选,ExecutionPolicy 才拥有 hard allow/deny 语义。两者 都被 Run pin,但 final authorization 必须重新解释 ExecutionPolicy 与 exact args。
class ExecutionPolicy(BaseModel):
schema_version: Literal["1"]
policy_id: str
stage_role_rules: tuple[StageRoleRule, ...]
principal_scope_rules: tuple[PrincipalScopeRule, ...]
required_config_rules: tuple[RequiredConfigRule, ...]
data_flow_matrix: tuple[DataFlowRule, ...]
approval_rules: tuple[ApprovalRule, ...]
argument_constraints: tuple[ArgumentConstraint, ...]
allowed_adapter_bundle_digest: str
effect_contract_rules: tuple[EffectContractRule, ...]
diagnostic_visibility_rules: tuple[DiagnosticVisibilityRule, ...]
output_policy: RuntimeToolOutputPolicy
max_tool_calls: int
max_result_tokens: int
max_wall_time_ms: int
max_cost_microunits: int
availability_ttl_seconds: int
stale_behavior: Literal["refresh", "wait", "deny"]
policy_digest: str
class StageRoleRule(BaseModel):
schema_version: Literal["1"]
rule_id: str
stage_ids: tuple[str, ...]
role_ids: tuple[str, ...]
capability_ids: tuple[str, ...]
side_effect_classes: tuple[str, ...]
disposition: Literal["ALLOW", "DENY"]
class PrincipalScopeRule(BaseModel):
schema_version: Literal["1"]
rule_id: str
principal_classes: tuple[str, ...]
capability_ids: tuple[str, ...]
required_scope_names: tuple[str, ...]
class RequiredConfigRule(BaseModel):
schema_version: Literal["1"]
rule_id: str
capability_ids: tuple[str, ...]
required_config_names: tuple[str, ...]
class DataFlowRule(BaseModel):
schema_version: Literal["1"]
rule_id: str
source_classes: tuple[str, ...]
destination_classes: tuple[str, ...]
capability_ids: tuple[str, ...]
disposition: Literal["ALLOW", "DENY"]
class ApprovalRule(BaseModel):
schema_version: Literal["1"]
approval_class: str
capability_ids: tuple[str, ...]
principal_classes: tuple[str, ...]
ttl_seconds: int
exact_argument_binding_required: Literal[True]
class ArgumentConstraint(BaseModel):
schema_version: Literal["1"]
constraint_id: str
capability_id: str
json_pointer: str
operator: Literal["EQ", "IN", "NOT_IN", "PREFIX", "MAX", "MIN", "MATCH_ENUM"]
operand: CanonicalJsonValue
class EffectContractRule(BaseModel):
schema_version: Literal["1"]
capability_ids: tuple[str, ...]
allowed_idempotency: tuple[str, ...]
allowed_checkpoint_policies: tuple[str, ...]
allowed_cache_policies: tuple[str, ...]
BlockedDiagnosticReason = Literal[
"AUTH_MISSING", "CONFIG_MISSING", "ENVIRONMENT_MISSING",
"PROVIDER_DEGRADED", "APPROVAL_REQUIRED", "BUDGET_NOT_ADMITTED",
]
DiagnosticRemediationClass = Literal[
"CLARIFY", "WAIT", "REQUEST_APPROVAL", "REQUEST_BUDGET",
"RETRY_PROVIDER",
]
class DiagnosticRemediationRule(BaseModel):
schema_version: Literal["1"]
blocked_reason: BlockedDiagnosticReason
remediation_class: DiagnosticRemediationClass
public_text: str
class DiagnosticVisibilityRule(BaseModel):
schema_version: Literal["1"]
rule_id: str
stage_ids: tuple[str, ...]
role_ids: tuple[str, ...]
principal_classes: tuple[str, ...]
capability_ids: tuple[str, ...]
blocked_reasons: tuple[BlockedDiagnosticReason, ...]
render_profile: Literal["IDENTITY_SUMMARY_REASON_REMEDIATION"]
remediation_rules: tuple[DiagnosticRemediationRule, ...]
disposition: Literal["ALLOW", "DENY"]
class RuntimeToolOutputPolicy(BaseModel):
schema_version: Literal["1"]
max_inline_bytes: int
max_model_tokens: int
artifact_required_above_bytes: int
structural_redaction_policy_digest: str
semantic_summary: Literal["forbidden", "separate_effect"]
renderer_digest: str上述 nested types 都是 closed、frozen models,按 canonical ID 排序;不允许
dict[str, Any] policy escape hatch。validator 至少检查 deny-overrides-allow、egress
闭包、approval class 存在、argument constraint 可判定、Adapter/effect contract 一致。
DiagnosticVisibilityRule 也是 policy_digest 的 canonical 内容;无匹配规则、多个
冲突匹配或 DENY 一律归为 HIDDEN_DENY。只有唯一 ALLOW 才可投影规则钉死的
namespaced identity、bounded summary、closed reason、remediation class/public text 和
descriptor digest;auth/config/scope 值、provider alias、schema、binding 与参数永不进入
diagnostic bytes。remediation 文本不得由模型、descriptor 或异常字符串临时生成。
policy.py 只实现一个 pinned、纯 normalize_effective_policy();selection eligibility
和 Runtime final authorization 都调用它。前者不给 exact arguments/可变 dispatch state,
只读取 argument-independent projection;后者以 exact canonical arguments、最新
Availability/Budget/Fence 再调用并独占授权提交权。两个调用共享 canonical rule matching、
reason precedence 和 normalized-policy digest,禁止各写一套 if/else。
Normative 合成规则:
| 输入约束 | Effective policy |
|---|---|
任一 matching DENY、descriptor incompatibility、frozen/workflow deny |
deny 胜出 |
| descriptor stage、Selection stage allowlist、Execution stage/role/capability/data allow | 取交集;缺 applicable Execution ALLOW 默认 deny |
| required scopes/config/approval/evidence/preconditions | 取并集,policy 只能增加要求 |
| allowed destination/content type/Adapter/effect/idempotency/cache class | 取交集 |
| max bytes/tokens/calls/wall time/cost | 取所有 applicable ceiling 的最小值 |
| network/output exposure | 取最严格值 |
| SelectionPolicy narrowing | 只能从 effective eligible set 删除,不能添加或授权 |
CapabilityDescriptor.stage_tags 不得为空;显式 "*" 表示 stage-neutral。stage 必须
同时满足 descriptor、SelectionPolicy 和 matching ExecutionPolicy ALLOW。缺 rule、
空 intersection、无法判定或冲突都 fail closed。统一 reason precedence 为:
integrity/schema
-> explicit deny
-> stage/role
-> data flow
-> Adapter/effect contract
-> scope/config
-> approval
-> availability
-> budget
-> dependency/trajectory
-> exact argument constraint
Run-admission ToolConfigurationValidator 还要拒绝 descriptor 要求无法被当前
ExecutionPolicy 表达、required approval class 无定义、以及
descriptor/Adapter/effect class intersection 为空的 bundle;Catalog compiler 本身仍
只验证 descriptor/source 内部一致性,不读取 Run policy。
validator 输出一个 canonical normalized_tool_configuration_digest,覆盖 catalog、
Selection/Execution policy、Selection Bundle、provider protocol、Adapter bundle、
normalized policy algebra 与兼容性矩阵。Run spec、Research Context binding、actor
request、outer Runtime Effect authorization 和 acceptance manifest 都绑定该 digest;
Catalog compiler 不能生成或冒充它。
contract tests 使用冲突矩阵证明 selection 与 final authorization 对共同输入得到相同
normalized policy/reason;final authorization 只能因 exact args 或更新后的 mutable state
增加更晚的 denial reason,不能重排或覆盖既有约束。
class AvailabilitySnapshot(BaseModel):
schema_version: Literal["1"]
generation: int
observed_at: datetime
expires_at: datetime
capability_status: tuple[CapabilityAvailability, ...]
attestation_refs: tuple[ArtifactRef, ...]
freshness: Literal["FRESH", "STALE"]
validated_at_event_seq: int
digest: strclass CapabilityAvailability(BaseModel):
schema_version: Literal["1"]
capability_id: str
status: Literal[
"AVAILABLE", "AUTH_MISSING", "CONFIG_MISSING", "ENVIRONMENT_MISSING",
"PROVIDER_DEGRADED", "DISABLED"
]
reason_codes: tuple[str, ...]
available_scope_names: tuple[str, ...]
available_config_names: tuple[str, ...]
provider_generation: str | None
attestation_ref: ArtifactRef | None
attestation_digest: str | None状态是 AVAILABLE | AUTH_MISSING | CONFIG_MISSING | ENVIRONMENT_MISSING | PROVIDER_DEGRADED | DISABLED。不存 secret 值。dispatch 时过期必须刷新或进入
typed wait,不能假定仍可用。Runtime 在调用 pure advance() 前计算并提交
freshness;ToolInteraction 不读取 live clock。final authorization 再检查当前状态。
views.py 实现 governing design 的 closed CatalogEntryView、CatalogView 和
InspectView。共同 validator 要求:
EligibilityDisposition = Literal[
"EXECUTABLE_ELIGIBLE", "BLOCKED_DIAGNOSTIC", "HIDDEN_DENY"
]
class BlockedCapabilityHintView(BaseModel):
schema_version: Literal["1"]
capability_id: str
summary: str
blocked_reason: Literal[
"AUTH_MISSING", "CONFIG_MISSING", "ENVIRONMENT_MISSING",
"PROVIDER_DEGRADED", "APPROVAL_REQUIRED", "BUDGET_NOT_ADMITTED"
]
remediation_class: str
descriptor_digest: str
class CatalogView(BaseModel):
schema_version: Literal["1"]
entries: tuple[CatalogEntryView, ...]
blocked_hints: tuple[BlockedCapabilityHintView, ...]
executable_token_count: int
diagnostic_token_count: int
catalog_digest: str
selection_policy_digest: str
renderer_digest: str
view_digest: strpolicy normalizer 必须先把每个 capability 分类为三种 disposition。permanent policy/
data/confidentiality/frozen-protocol deny 只能是 HIDDEN_DENY,在 actor view 中完全
消失。只有可补救的 auth/config/environment/provider/approval/budget 缺口可进入
BLOCKED_DIAGNOSTIC。executable 与 diagnostic 两个 lane 各自有独立 budget、stable
rank、trace 和 digest;execution ranker 只能排序 EXECUTABLE_ELIGIBLE,diagnostic
ranker 只能排序 approved hints。blocked hint 不含 schema、provider alias、binding、
arguments 或 auth scope,Inspect/Execute/fallback 都只接受 entries 成员。命中 hint
只能产生 clarify/wait/remediation,永远不能准备 Effect 或被提升为 executable。
- entry 按 capability ID stable sort;schema/examples/dependency 保持 declared order;
- 每个 view 绑定 catalog、SelectionPolicy、SelectionBundle renderer、descriptor/schema digests、exact tokenizer count 和自身 digest;
- Catalog 的 executable entries 只含 identity/summary/availability/risk/full-schema-token estimate;Inspect 才含一个 capability 的 full schema/examples/preconditions/dependencies;
DISCOVER输入绑定旧 view/query digest,输出下一 model Effect 的新 view/toolset digest;不能在同一 prepared Effect 中热换 schema;- Catalog/Inspect/Execute 只是 internal view phases,不注册为 model-callable tools。
interaction.py 的 actor request 与 runtime/adapters/tool_interaction.py 的运行时
authority 必须是两个不相互嵌套的 closed model:
class ActorTurnScope(BaseModel):
schema_version: Literal["1"]
actor_request_scope_digest: str
actor_workflow_view_digest: str
actor_continuation_view_digest: str
stage_name: str
actor_stage_contract_view_digest: str
actor_decision_contract_view_digest: str
actor_interaction_key: str
decision_epoch: int
class RuntimeAuthorizationEnvelope(BaseModel):
schema_version: Literal["1"]
run_id: str
run_spec_digest: str
full_evaluation_contract_digest: str
activity_id: str
activity_generation: int
expected_event_seq: int
fencing_epoch: int
decision_intent_id: str | None
decision_intent_digest: str | None
actor_request_digest: str
actor_decision_slot_digest: str | NoneToolInteractionRequest 只含 ActorTurnScope、actor-visible Research Context/
Decision Input/Decision Memory、pinned catalog/policies/bundle、availability、budget、
rollout 和 trigger。它禁止 Run/Activity/fence/full private Evaluation Contract、actual
Decision Intent ID、evaluator/treatment metadata。DecisionMemoryBinding 使用
actor_decision_slot_digest 与 optional parent outcome actor-projection digest;actual
Intent 只存在 envelope。Runtime journal 以 append-only link 把 actor request/directive
digest 与 envelope 绑定。actor transcript cache 只绑定 actor scope;Effect identity、
execution authorization/cache 绑定 actor digest + envelope,禁止跨 Run 复用 Effect。
closed continuation types 至少包括:
ToolInteractionTrigger = (
BeginInteraction | ModelEffectSettled |
ModelPreparationRejected | ModelDispatchAuthorizationRejected |
ToolEffectsSettled | ToolPreparationRejected |
ToolDispatchAuthorizationRejected |
InputResolutionCommitted | DecisionTurnInputsCommitted
)
ToolDirective = (
PrepareEffect | CommitDecisionAndRequestNextTurnInputs |
CommitStageProposal | WaitForInput | RejectActivity
)
class NextTurnInputSpec(BaseModel):
schema_version: Literal["1"]
actor_interaction_key: str
next_decision_epoch: int
cause_digest: str
committed_trajectory_digest: str
decision_input_projection_spec_digest: str
catalog_or_toolset_view_digest: str
decision_point: DecisionPoint | None
memory_profile_digest: str | None
parent_recall_outcome_required: bool
spec_digest: strDISCOVER、ToolFeedback、input resolution、model protocol correction、tool
preparation rejection 和 tool dispatch rejection 后,需要 model continuation 时的
第一次 advance() 只能返回
CommitDecisionAndRequestNextTurnInputs:关闭旧 Tool Decision/Recall Outcome 并描述
下一轮输入,不创建 Effect。Runtime 提交/link 旧 outcome,生成 bounded Decision
Input,预分配 child Intent/actor slot,重新 recall 或创建 NOT_REQUESTED binding,并
删除旧 rendered Recall bytes;然后以 DecisionTurnInputsCommitted 第二次调用
advance()。只有第二次调用可 PrepareEffect(model)。任一中间 crash 只补 missing
suffix,不能复用旧 Intent/Recall 或重复创建 model Effect。
NextTurnInputSpec.cause 与 trigger 有 closed applicability matrix;例如 discovery 只能
使用 DISCOVERY,tool rejection 只能使用对应 rejection cause。Model Effect rejection
不适用此立即续回合规则:它按下一段关闭当前 Intent 并 wait/reject,只有后来真正的新
committed state 才能从 InputResolution/availability resolution 进入两步协议。
Runtime 构造 source_status=NOT_REQUESTED actor binding 前,必须 read-back actual
Decision Intent 的 admission-proof-backed Memory Governance Binding 并确认
recall_requirement=REGISTERED_NOT_REQUESTED;REQUIRED Intent、model proposal 或
ToolInteraction 均无权把它降成 not requested。
model 与 tool rejection 是不同 closed branches。ModelPreparationRejected 表示
exact model request/provider/data-flow/budget 在 Effect 创建前被拒;
ModelDispatchAuthorizationRejected 保留已创建 model Effect/Invocation、
not-executed receipt/reason 与 settlement。二者都不构造 ToolFeedback,只能按 pinned
policy 关闭当前 Intent 后 WaitForInput/RejectActivity;若 operator/input/provider 的新
committed state 允许再试,必须走 next-turn two-call protocol 并使用新 Intent,不能作为
hidden Adapter retry。
data_flow.py 定义 closed、canonical types:
class DataClassification(BaseModel):
schema_version: Literal["1"]
confidentiality: Literal["PUBLIC", "INTERNAL", "CONFIDENTIAL", "RESTRICTED"]
compartments: tuple[Literal["EVALUATOR_PRIVATE", "CREDENTIAL_SECRET"], ...]
integrity: Literal["VALIDATED", "UNTRUSTED_EXTERNAL"]
DataDestinationClass = Literal[
"LOCAL_PROCESS", "TRUSTED_MODEL_PROVIDER", "RESTRICTED_NETWORK", "OPEN_WORLD"
]
class DataProvenanceBinding(BaseModel):
schema_version: Literal["1"]
source_kind: Literal[
"DECISION_INPUT", "RECALL_CARD", "TOOL_FEEDBACK", "ARTIFACT",
"USER_INPUT", "TRUSTED_CONSTANT"
]
source_ref: ArtifactRef | None
source_digest: str
classification: DataClassification
declassification_receipt_digests: tuple[str, ...]
class ArgumentSourceClaim(BaseModel):
schema_version: Literal["1"]
json_pointer: str
claimed_source_digests: tuple[str, ...]
class DeclassificationReceipt(BaseModel):
schema_version: Literal["1"]
source_digest: str
output_digest: str
source_classification: DataClassification
output_classification: DataClassification
removed_field_paths: tuple[str, ...]
policy_digest: str
trusted_transform_effect_id: str
receipt_digest: strjoin 算法固定为 confidentiality 取最大级别、compartments set-union + stable sort、
任一 untrusted 输入令 integrity 为 UNTRUSTED_EXTERNAL。model 的 per-pointer claim
只能增加来源,不能删来源或降级;所有自由生成参数默认继承当前 actor-visible
Decision Input、Recall、ToolFeedback 与 artifact 的并集。只有 trusted deterministic
transform Effect 的 verified derivation/declassification receipt 可收窄,且
EVALUATOR_PRIVATE/CREDENTIAL_SECRET 不得 declassify 到 actor/open-world。
DecisionInputView、DecisionMemoryBinding、ToolCallProposal、raw receipt、artifact、
CommittedToolReceiptView 和 ToolFeedback 都带 classification/provenance digest。
ModelEffectSpec 与 ToolEffectRequest 都必须绑定 destination class、effective input
classification、input/argument provenance digest、declassification receipt digests、
execution-policy digest 和由 Runtime 重算的 data_flow_digest。model provider、semantic
summarizer、renderer/truncator 与 tool-to-tool forwarding 都走相同 egress matrix;不能
把摘要/截断当成洗白边界。Runtime 在 Effect preparation 和原子 dispatch transaction
各重算一次,未通过前不得发送 provider/tool bytes;output classification 是 effective
input 与 descriptor output class 的 join。
tracing.py 的记录必须覆盖:
- request/turn/trajectory/catalog/selection-policy/Selection-Bundle/ availability/budget digests;
- actor decision-slot、Recall Input Snapshot content、optional Recall Context actor projection digests,或显式 no-recall;actual Decision Intent/Run/Activity/private contract identity 只由 Runtime envelope/link event 保存;
- active/shadow lane;
- candidate universe;
- 每个 capability 的 hard-filter disposition/reason;
- query artifact/digest;
- lexical/dense/fused ranks、score 与 ranker digest;
- exposed ordered capabilities、provider aliases、schema digests/tokens;
- model proposal interaction/control、arg digest;
- proposal validation disposition;
- correction/discovery counters;
- resulting interaction/control digest,
EXECUTE另含 exact capability 与 canonical-argument digests; - redaction policy digest。
每次 advance() 恰好生成一个 step-level Tool Decision Record。它不包含因果上稍后
才知道的 directive、authorization、Effect、Physical Invocation 或 memory outcome
ID。后续分别由 ToolDirectiveEmitted、ToolAuthorizationRecorded、
ModelPreparationRejected、ModelDispatchAuthorizationRejected、
ToolPreparationRejected、ToolDispatchAuthorizationRejected、ToolEffectLinked、
普通 invocation/receipt events 和
MemoryToolDecisionLinked append-only 关联;trace 只是 join projection,不能回写
原 record。所有集合 canonical sort;所有有语义的 wire order 单独保存,不依赖 map
insertion order。
feedback.py 按 governing design 的 closed status 生成 bounded view。full output
先完成 digest、artifact commit、malware/content policy 与 redaction,再构造模型
视图。摘要失败不能丢失 raw receipt;按 pinned policy使用结构化截断或 wait/fail。
Runtime Adapter 先构造 immutable CommittedToolReceiptView,再交给 pure
ToolInteraction。需要语义摘要时另建受预算 Model Effect;不得在 feedback.py
隐式调用模型或读取 artifact。closed status 是:
OK | EMPTY | RETRY_EXHAUSTED | PERMANENT_FAILURE | CONFIRMED_NOT_EXECUTED
LEGACY_UNTYPED
DENIED | UNAVAILABLE | BUDGET_NOT_ADMITTED | APPROVAL_REQUIRED
前两行中的 Effect-backed status 必须有 Effect ID、Physical Invocation ID、selected
receipt、classification/provenance digest 和 settled usage;其中
RETRY_EXHAUSTED 只在 Runtime 已耗尽其唯一 retry policy 后生成,并绑定
runtime_retry_disposition=EXHAUSTED。Adapter 的中间 retryable 事实不直接进入
ToolFeedback。CONFIRMED_NOT_EXECUTED 还必须有 closed
dispatch_rejection_id/reason、Adapter 未进入证明及 reservation release/settlement;它
保留已有 Effect/Invocation lineage,并由 ToolDispatchAuthorizationRejected 触发。
最后一行必须有 ToolPreparationRejected ID,且 Effect ID、Physical
Invocation、artifact output、settled usage 均为空。closed validator 保证二者恰有其一,
并校验 NOT_APPLICABLE | EXHAUSTED | NOT_RETRYABLE retry disposition 与 status
组合。
LEGACY_UNTYPED 只允许迁移期,并在 T8 前归零。每个 legacy tool 必须有 fixture
将已知 success/error/empty shapes 映射为 typed evidence;未知 shape 不能默认为
成功。
interaction.py 公开给 Runtime Activity Adapter 的唯一入口:
class ToolInteraction(Protocol):
def advance(self, request: ToolInteractionRequest) -> ToolDirective: ...contract tests 通过同一个 Interface 测 static、shadow、hybrid 和 trajectory policy。
不公开 search() / inspect() / authorize() / execute() 给 Agent 或应用 caller。
interaction.py 必须逐字段实现 governing design 的 bounded DecisionInputView、
actor-safe DecisionMemoryBinding、ActorTurnScope、single PrepareEffect、
CommitDecisionAndRequestNextTurnInputs、NextTurnInputSpec、
DecisionTurnInputsCommitted、ModelEffectSpec、ToolEffectRequest、
ModelPreparationRejected、ModelDispatchAuthorizationRejected、
ToolPreparationRejected、ToolDispatchAuthorizationRejected 和 acyclic digest order;不得以
dict[str, Any] 或 pass-through kwargs 代替这些 contracts。
RuntimeAuthorizationEnvelope 只能定义/持久化在 Runtime side。dependency/serialization
tests 禁止它或其中任一 Run/Activity/fence/private-contract/actual-Intent 字段出现在
ToolInteractionRequest、provider bytes、actor turn contract、Tool Decision Record 或
actor cache key。
Runtime Activity Adapter 在进入循环前解析并验证 Selection Bundle,把 tokenizer/
ranker/index/renderer 作为 read-only constructed dependencies 注入 ToolInteraction
Implementation;advance() 期间不得 lazy-load、联网或读取 mutable process cache。
为了不一次重写 MetaChain.run_async(),T3 新增 private
LegacyMetaChainToolAdapter:
class LegacyMetaChainToolAdapter:
def open(self, request: LegacyInteractionOpen) -> LegacyInteractionSession: ...
class LegacyInteractionSession:
@property
def active_cache_identity(self) -> dict[str, Any]: ...
def prepare_turn(
self,
active_agent: Agent,
history: list[dict],
turn_index: int,
) -> LegacyPreparedTurn: ...
def resolve_turn(
self,
prepared: LegacyPreparedTurn,
model_message: Message,
context_variables: dict,
) -> LegacyResolvedTurn: ...它是短生命周期 Adapter,不持久化 Python closure。durable record 只保存 canonical ID、artifact ref、schema/policy/Adapter digest 和 closed data type。replay 时重新 构造 Adapter。
T3 AgentModule 的 caller 变化应限制为:创建 Adapter/session、把 active cache
identity 传给 CacheIdentity、把 session 交给 MetaChain。当前 Agent factory 和
run_infer_* 不需要在同一个 PR 改写。
legacy-v1 profile 必须复用当前 callable schema 生成、排序、provider-specific
tool choice、parallel flag、message conversion、return conversion、context patch 和
handoff。shadow-v1 不增加 mode-specific synchronous serialization 或 Run-journal
event。active model request artifact 提交后,独立 non-causal audit projector 从该
immutable artifact 派生 bounded shadow-outbox ref,写入不递增 Run event sequence 的
audit namespace。独立 worker 随后做 selection-only replay;active path 不等待
projector/worker,也不在 critical path 运行 ranker。
Projector 以 committed request artifact cursor 扫描并 read-back,shadow job ID 由
request digest + shadow policy digest 派生;crash 后补扫、重复通知和重复 claim 均只
产生一个 logical shadow record,cursor/outbox 永不写回 active journal。
Golden assertion:同一 fixture 在 legacy-v1 与 shadow-v1 下,除 shadow
audit projection/outbox/result 外,下列 active 内容逐字节一致,Run event sequence 与
active Budget Envelope 不变:
- system/user/history messages;
- provider-visible tool schema 和顺序;
- effective
tool_choice与parallel_tool_calls; - model request/response digest;
- actual tool/control calls、args、返回和顺序;
- context patch、handoff 和 stage result;
- active cache identity/hit;
- active Effect/Physical Invocation(durable path 上线后);
- total provider calls。
Shadow failure 记录 SHADOW_SKIPPED | SHADOW_FAILED,不能让 active Activity 失败。
production selection-only shadow 禁止 remote ranker 和 counterfactual model call。
shadow 使用隔离 worker/resource quota;其可能的资源争用不宣称为零,而由正式结果前
冻结的 p50/p95 active-latency overhead SLO 约束。
需要完整 DIRECT/EXECUTE 反事实时启动独立 replay Research Run,使用独立 budget、
journal、cache/namespace 与 recorded-read/isolated deterministic tool fixtures;不
重复 live irreversible Effect,结果不回写 active Run。
固定 profile:
legacy-v1
-> shadow-v1
-> dynamic-required-v1 # 只缩 capability;保留 legacy control closure;tool 经 Runtime Effect
-> dynamic-auto-v1 # typed interaction/control;允许 DIRECT/CLARIFY
-> durable-v1 # typed output/Adapters 收敛,LEGACY_UNTYPED=0
这些名字是 immutable artifact 实例,不是 Python subclass:
class ToolUseProfile(BaseModel):
schema_version: Literal["1"]
profile_id: str
selection_mode: Literal["legacy", "static", "shadow", "adaptive"]
selection_policy_ref: ArtifactRef
selection_policy_digest: str
selection_bundle_ref: ArtifactRef
selection_bundle_digest: str
provider_protocol_ref: ArtifactRef
provider_protocol_digest: str
control_mode: Literal["legacy_tools", "typed"]
execution_mode: Literal["legacy_callable", "durable_effect"]
shadow_mode: Literal["off", "selection_outbox"]
fallback_scope: Literal["original_stage_allowlist", "none"]
profile_digest: strProfile validator 强制 selection_mode=adaptive -> execution_mode=durable_effect。
legacy_callable 只允许 legacy-v1 与 shadow-v1 active lane;不能生成 canonical
ToolFeedback、authorization incident 或 DISPATCHED 证据。T4/T5 的 legacy output
binding 仍使用 durable_effect,只是 receipt projection 暂标 LEGACY_UNTYPED。
dynamic-required-v1 只用于短期兼容 canary:
- capability 候选可缩小;
- legacy
case_*/transfer_*control closure 始终保留; - 低置信度 fallback 只回到该 Agent 原始 allowlist,不扩大到全局 Registry;
- 不改变无 tool call 时的 legacy 行为。
dynamic-auto-v1 只有在该 stage 的 typed completion/wait/handoff contract 通过后
才启用,并在同一 stage 删除无限 “Please use the tools” correction。不得全局一次性
将 required 改为 auto。
每个 Phase 可拆多个小 PR,但不得跨越 exit gate。每个 PR 必须增加通过目标 Interface 的测试,且旧测试保持绿色。
修改
- 扩展
evals/trace.py,增加ToolAttemptTrace,LegacyToolDecisionProjection,ToolExposureTrace,ToolFeedbackTraceclosed types; MetaChain.handle_tool_calls()在 name lookup、JSON parse、schema validate、call start、exception、return conversion 每个 seam 写完整 attempt;- 同步与 async 路径使用同一个 tracing Implementation;
ToolModule从 typed outcome 计算 success,不再硬编码;FlowModule.export_runtime_trace()包含 Agent 内 model/tool/control call;- 给所有 trace 增加 run/stage/agent/turn/trace IDs 和 schema token count;
- 生成 baseline capture/analyze 命令和 manifest;
- 添加 provider request spy 与 deterministic fake model/tool fixtures。
Baseline “完整 request/args/return”不等于明文落盘:捕获前先做 data classification;credential/secret 字段拒绝持久化,普通 trace 只留 typed redaction、 digest 与 policy ref;确需复现的敏感 payload 进入加密、访问控制、有限 retention 的 artifact namespace。任何 evaluator-private label/answer 都不得进入 actor trace。
T0 类型只是 legacy 事实投影,不是 canonical Tool Decision Record。T2 建立
tool_use/tracing.py 后,Runtime journal 才成为 mutation authority,T0 projection
迁为对 canonical events 的只读 Adapter。
不得改变
- schema、tool order、tool choice、prompt、turn termination、tool execution、cache 或 Agent handoff 语义。
Exit gate T0
- fixture 中每个 provider tool proposal 恰好对应一条 attempt terminal disposition;
- unknown/parse/exception/error-string/empty cases 都不被记录为 success;
- 两个 entry flow 的 frozen fixture trace 可重放并自动重算工具数/tokens/latency;
- 26 个空 log 或缺失
trace.json的旧现状不再出现在新 fixture; - baseline manifest 在 adaptive 代码进入前归档并固定 digest。
新增
tool_use/catalog.py的独立CapabilityCatalogCompilerModule/contracts;LegacyRegistryCatalogAdapter与SkillManifestCatalogAdapter;catalog check/build/diff命令;- inventory report:decorator、manifest、Agent wrapper、active stage references;
- schema/binding/identity conflict 和 required config validation;
- content-addressed catalog artifact。
Compiler 只由 Run admission/Snapshot build 调用;ToolInteraction 只能消费已验证 artifact,不能 import source Adapter、读 manifest 或访问 Registry。
迁移规则
- 先为所有 active Agent callable 生成
legacy-incompletedescriptor; - 每个 descriptor 在 T4 前补齐 use/non-use、risk、effect、data、output、dependency policy;
legacy-incomplete只允许 static/shadow,不允许 adaptive enforcement;- manifest 参数 schema 与 callable/typed request schema 比较,不再成为平行真相;
SkillRegistry.register_skill()不再允许同名后注册者静默覆盖生产 binding;- 删除 search 初始化过程的 MD5 fallback;在 T5 前可以保留显式 lexical-only test degradation。
Exit gate T1
- 当前 active tool universe 100% 有 namespaced identity、schema 和 execution binding;
- catalog compile 无 silent import failure、name collision 或 schema mismatch;
- 每个 current Agent 的 ordered callable list 可从 static policy + catalog 精确重建;
- catalog 在随机 map insertion order 下 digest 稳定;
- manifest 与 legacy Adapter 运行同一 contract suite;
catalog diff能解释每个行为相关 digest 变化。
新增
interaction.py的唯一advance()Interface 和 closed request/directive types;views.pyboundedDecisionInputView、executable/blocked-diagnostic Catalog/Inspect 与 committed receipt views;- actor-safe
ActorTurnScope/decision-slot bindings、Runtime-onlyRuntimeAuthorizationEnvelope与 append-only journal linkage; - 两步
CommitDecisionAndRequestNextTurnInputs -> DecisionTurnInputsCommittedcontinuation 和 crash-recovery contract; data_flow.pyclosed classification/provenance/declassification/join/egress contracts;bundle.pystatic Selection Bundle 与 verifier;policy.pystatic parity policy artifact 与 pinned interpreter;selection.pydeterministic eligibility 和 small-set bypass;protocol.pycanonical interaction/control proposal;feedback.pytyped/bounded projection;benchmark/tool_use分离的 actor/evaluator-private case schema、runner 和 analyzer;- deterministic fake catalog/model/tool/evaluator。
首批 case family
- direct、clarify、single、跨 turn 顺序 multi-tool;
- local file vs web/GitHub/arXiv confusion;
- auth/config/environment missing;
- write/destructive/egress denial;
- empty/transient/permanent/ambiguous;
- long output、prompt injection;
- memory-informed tool choice、memory abstention、malicious Evidence Card、 evaluation-plane memory isolation;
- dependent order、provider multi-call proposal 必须按 v1 single-call contract 拒绝;
- repeated failure 和 evidence satisfied。
Exit gate T2
- 相同 canonical request + Selection Bundle digest 产生 byte-identical directive/digest;
- dense-enabled bundle 精确绑定 catalog/descriptor corpus/numeric runtime;陈旧、 partial、catalog-swap 或 worker mismatch 不得进入 active selection;
- 任一 model-visible schema/order/alias/policy/protocol/Adapter change 改变正确 digest;
- 无关 metadata 和 dict insertion order 不改变 digest;
- closed interaction/control combination 全覆盖;
- forbidden candidate 永不进入 ranker;
- schema budget/low-confidence/degradation 行为由 policy fixture 精确测试;
- ToolFeedback 超预算时生成 artifact ref + bounded view;
- tool final-authorization denial 经
ToolPreparationRejected回流且 Effect/Invocation=0; - blocked-diagnostic hint 只产生 clarify/wait/remediation;无 schema/alias/Effect,
HIDDEN_DENYactor bytes 为 0; - actor request/cache 对 Run/Activity/fence/actual Intent/private contract 变体逐字节不变, outer Runtime authorization/Effect identity 随 envelope 改变且不能跨 Run replay;
- DISCOVER/feedback/input/protocol/rejection continuation 必须先 commit old outcome/new
input spec,第二个
advance()才能 prepare model Effect;中间 crash 不重复 outcome/ Intent/Effect; - data classification join、model claim only-add、trusted transform declassification、 model/tool egress fail-closed 与 output inheritance 均有 golden/property tests;
- evaluator label type/bytes 无法进入 ToolInteraction、actor prompt、turn/cache identity;
- 同 actor fixture/view 改变 private labels 或 full evaluator contract digest 时,actor request/turn/cache bytes 不变;
- benchmark raw trace 可独立重算全部 diagnostic metrics。
新增/修改
- private
LegacyMetaChainToolAdapter; AgentModule把 session active identity 传给 cache;MetaChain每轮调用 migration Adapter 的 prepare/resolve;- 所有 Agent factory 绑定
legacy-v1/shadow-v1profile ref; - non-causal audit projector、bounded shadow outbox、selection-only shadow record 和 independent replay command;
- active/shadow lane isolation;
- shadow inertness golden test 覆盖 sync/async 与 provider profiles。
选择 Implementation
T3 projector 只从已提交 canonical actor request artifact 派生 outbox ref。独立 worker 运行 hard filter + static allowlist + small-set bypass,记录 candidate/exposure/need-gate prediction;它不 预测 counterfactual model response。dense/hybrid 或完整 DIRECT/EXECUTE 反事实只在 独立 replay Run 中执行,不能在 active critical path 隐式加载模型或 Chroma。
Exit gate T3
- 两个 entry flow、所有 active Agent 均产生 shadow record;
- legacy 与 shadow active lane byte-equivalent;
- shadow failure/timeout 不影响 active result;
- Run event sequence 与 active Budget Envelope 不含 shadow execution;
- active path 不等待 shadow,且 p50/p95 active-latency overhead 通过预注册 SLO;
- active cache hit 不受 shadow policy/digest 影响;
- selection replay 只使用捕获的 canonical input并得到同一 shadow digest;
- 候选 universe/filter/exposure/need-gate prediction 覆盖率可自动报告。
新增/修改
NativeFunctionCallProtocolAdaptercanonical round-trip;- provider capability matrix 与 prompt-emulation contract tests;
- stage completion/wait/handoff typed contract;
- control namespace 与 legacy control compatibility mapping;
- 首个 read-only stage 切
dynamic-required-v1,然后切dynamic-auto-v1; - 该 stage 的 Agent tool list 改为 catalog + policy 生成;
- 低置信度 static-stage fallback,不允许 global expansion;
- run spec/cache/trace 绑定 active policy/Selection-Bundle/toolset/control digests。
- read-only legacy binding 通过 Runtime Effect Adapter 执行并产出真实 Effect/Invocation/receipt/settlement;禁止回退 direct callable。
- Runtime preparation 只创建
DISPATCHABLEEffect/Invocation/reservation;紧邻 Adapter entry 的 dispatch transaction 重验 state/generation/event/fence/incident、 exact bindings、live auth/config/availability/approval、data-flow、budget/reservation 与 effect-kind memory lineage,成功后才提交DISPATCHED:model Effect 校验当前 Intent/actor slot/Recall Input/optional Context 且该 turn 尚无 terminal Outcome;只有 settledEXECUTEproposal 的 tool Effect 才要求 terminal Recall Decision Outcome 与MemoryToolDecisionLinked。
Canary 顺序
- 已使用 optional tool choice 的只读 survey/idea role;
- coding-plan read/planning role;
- 其他无 write/command/Browser capability 的 role。
Exit gate T4
- canary 正确产生 direct 和 clarify,不进入 required correction loop;
- control action 不出现在 capability catalog、Effect 或 tool-call metrics;
- model 不能调用未暴露 alias、stale toolset 或非法 interaction/control combination;
- forbidden exposure/call 和 sensitive egress 为 0;
- 每个 executed canary call 有真实 Effect/Invocation/selected receipt/settlement;
- prepare 后 mutable authorization 变化必须落
CONFIRMED_NOT_EXECUTED、settle reservation、零 Adapter entry,并经ToolDispatchAuthorizationRejected回流; - 与 static baseline 比较的 end-state gate 非劣;
- fallback 次数、原因和 schema budget 全部可解释;
- scientific/performance rollback 仅改变新 Run admission,健康 in-flight Run 保持
pinned profile;integrity/security trigger 必须按 §9.3 建立 incident、revoke fence,
并使受影响 scope 的新
DISPATCHED为 0。
新增
- deterministic lexical ranker;
- pinned local dense ranker Implementation 与 immutable model/index assets;
- deterministic fusion、stable tie break、adaptive top-k;
- tool-level positive/negative/pseudo-use queries 与 confused-with metadata;
- closed Catalog/Inspect view models、deterministic renderer/token accounting;
- low-confidence expand/clarify/static fallback;
- executable 与 blocked-diagnostic 两 lane 的独立 rank/budget/digest;
- committed ToolFeedback/subgoal/availability/evidence-gap re-selection;
- dense-only/static/hybrid/trajectory ablation runner。
约束
- 不新增 remote router 作为 v1 前置;
- dense 初始化失败按 policy 显式
lexical_only | expand | clarify | wait | fail; - 小集合 schema 在预算内时不调用 retriever Effect;
- 同一个失败 proposal 无新状态/参数变化不得无限重复;
- active toolset 只在新 committed trigger 后变化。
- every re-selection/model continuation 使用两步 next-turn input commit,不能在旧 Decision Intent/Recall binding 上直接创建 child model Effect。
Exit gate T5
- over-budget case 的 exact exposed schema 不超过 pinned budget;
- small-set bypass case 不增加 remote/model selection call;
- hard-negative 与 multi-tool set Recall/precision 报告完整;
- empty/typed failure 后能选择替代/扩展/澄清,不重复相同失败循环;
- evidence satisfied 后可转 DIRECT,不做多余工具调用;
- adaptive arm 通过 preregistered end-state noninferiority gate 后才能发布效率改善。
前置
T4 的 read-only Runtime Effect vertical slice 已通过;Durable Runtime Phase 2 的 Effect journal、Physical Invocation、Budget Envelope、fencing、reconcile、artifact 与 one-retry-owner contracts 保持绿色。
新增/修改
- 把 T4 的
runtime/_tool_authorization.pysealed authorization 与runtime/adapters/tool_interaction.pyvertical slice 扩到全部 stage; - 补齐 in-process/subprocess/Docker/Browser/fault ToolExecutionAdapters;
- exact capability/schema/binding/args/toolset/policy/Selection-Bundle digest validation;
- dispatch transaction 内 availability/approval/data-flow/budget recheck;
DISPATCHABLE -> DISPATCHED是唯一 Adapter-entry gate;失败提交CONFIRMED_NOT_EXECUTED/settlement 与ToolDispatchAuthorizationRejected;- full output artifact commit 与 bounded typed ToolFeedback;
LEGACY_UNTYPEDlegacy binding/string output 逐个迁到 typed execution binding 并归零;- 禁止 tool/SDK hidden retries;
- flow-level
ToolModule和 Agent internal tools 汇合到同一 Effect path。 - v1 provider proposal 最多一个 external tool call;multi-tool task 在 committed ToolFeedback 后由下一 turn 继续;
ToolPreparationRejectedclosed trigger 和无 Effect-ID ToolFeedback;- model/tool Effect 共用 provenance/classification/egress authorization;
- memory-informed proposal 的 Recall Decision Outcome/link guard。
Exit gate T6
- 每个 external dispatch 有唯一 Effect 与 Physical Invocation/reservation;
- denial 路径 logical Effect 与 Physical Invocation 数均为 0;
- dispatch rejection 保留已有 Effect/Invocation、Adapter entry=0、reservation 已 settle;不得误算为 preparation denial;
- denial/unavailable/budget/approval 能回流 reselect/wait/fail,不悬挂等待不存在的 receipt;
- stale fence、stale toolset、schema swap、args tamper 均不能
DISPATCHED; - crash matrix 在 prepare/dispatch/return/receipt/feedback/reselect seams 全过;
- known/not-executed/ambiguous/reconciled/unresolved semantics 与 durable design 一致;
- late receipt 只能进 evidence inbox;
- unknown outcome 永不成为 successful ToolFeedback;
- memory-informed tool invocation 在 Recall Decision Outcome link 前
DISPATCHED=0; - in-process/subprocess/Docker/Browser Adapter contract tests 全过;
LEGACY_UNTYPED数量按 manifest 跟踪并持续下降。
修改
- 按 read-only -> planning -> external search -> write/command/Docker/Browser -> judge/control 顺序迁移;
- Agent factory 只引用 stage capability/completion policy,不决定 callable list;
case_*/transfer_*从 Tool Catalog 和 provider capability list 删除;MetaChain.handle_tool_calls()生产执行职责删除,只保留 protocol Adapter 所需 的 wire translation 或整体被替换;FlowModule.ToolModule不再是生产 execution/cache/retry authority;- 两个 entry flow 通过相同 native Stage/Tool Interaction Adapter;
- Tool Effect trace 从 Runtime journal 投影到 Evaluation Contract; -所有 legacy string return/tool wrappers 迁为 typed evidence。
Exit gate T7
- 所有 active Agent stage 都有 dynamic-auto/durable profile 或经批准的 no-tool control-only profile;
- concrete tool import 不再决定 Agent 暴露集合;
- direct callable dispatch 的 production caller 为 0;
LEGACY_UNTYPED为 0; -两个 entry flow 的 parity、cache isolation、restart、completion/handoff tests 全过;- final Research Run completion 仍要求 Verification/Experience、required Recall Decision Outcome、Experience Distillation Receipt 与 operational settlement 完整链。
执行
- static/hard-filter/dense/hybrid/trajectory 正式 Evaluation Contract;
- hard-negative catalog scale sweep;
- provider profile matrix;
- fault injection 与 repeated-trial reliability;
- shadow -> read-only canary -> risk canary -> percentage rollout -> default;
- rollback drill;
- acceptance manifest 与 raw artifacts 归档。
删除/收敛
- production
legacy-v1和LegacyToolExecutionAdapter; Agent.functions作为 capability selection authority;- required-tool endless correction;
case_resolved,case_not_resolved,transfer_to_*capability definitions;SkillRegistry向 mutable global Registry 注入/覆盖;- recursive eager tool import;
- public/production
SkillRegistry.search_tools()和 MD5 fallback; ToolModule独立 execution/cache/retry/trace authority; -重复的 tool schema truth sources;- migration session Adapter。
Exit gate T8
- Section 10 的 release gates 全部通过;
- deletion
rg、import graph 和 sealed legacy harness tests 证明生产无旧 caller; - rollback 使用 retained durable bundle 或关闭 admission,不重新启用被删旧 authority;
- governing design 可从 Proposed 改为 Accepted;本计划可改为 Implemented。
Interface 是测试表面。测试不得依赖 private ranker 的中间 class 名或调用次数,除非 该次数本身属于 pinned Budget/Effect contract。
tests/test_tool_use/test_interaction_contract.py 必须覆盖:
- canonical request determinism 与 directive digest;
- 相同 request 在不同 Selection Bundle digest 下不得共享 decision/cache;
- request 包含 bounded canonical
DecisionInputView/receipt view,advance()不做 artifact I/O、clock read 或 hidden summarization; BeginInteraction -> model正常路径;- DIRECT、CLARIFY、DISCOVER、EXECUTE;
- CLARIFY wait suspend/resume 保持 Interaction ID,InputResolution 后先返回
CommitDecisionAndRequestNextTurnInputs并递增 decision epoch;只有带已提交 child inputs 的第二次调用才能 prepare model Effect;stage/subgoal/handoff 才创建新 Interaction; - 每个 memory-governed model Effect 使用新 Decision Intent;DISCOVER、CLARIFY、 ToolFeedback 后的 child turn 不保留旧 rendered Recall Context bytes,并绑定 parent Recall Decision Outcome digest;
- DISCOVER、ToolFeedback、InputResolution、protocol correction、tool preparation/ tool dispatch rejection 全部通过相同两步 continuation;model rejection 先 wait/reject; 在 outcome commit、child Intent/input/ recall preallocation 每个 seam crash/replay 都只补 missing suffix;
- actor request/directive 不含 Run/Activity/fence/full private contract/actual Intent; Runtime journal envelope binding 可 read-back 且 outer Effect identity 不跨 Run;
- interaction × control 的全部合法/非法组合;
- discovery/correction step 上限;
- same failed call loop detection;
- exact toolset/turn-contract echo;
- unknown/not-exposed/stale alias/invalid args;
- wait/fail closed errors;
- COMPLETE 不直接生成 Run terminal;
- HANDOFF 只允许 pinned workflow role;
- uncommitted/stale/ambiguous receipt 不进入下一次 trajectory;
- artifact-backed ToolFeedback 和 inline token budget。
- 每次
advance()恰好一个 Tool Decision Record,且无未来 directive/auth/Effect/Invocation/memory-outcome fields; ToolPreparationRejected的 denial/unavailable/budget/approval -> reselect/wait/fail,四类均 Effect/Invocation=0;ToolDispatchAuthorizationRejected->CONFIRMED_NOT_EXECUTED保留 Effect/Invocation、 settlement 完整、Adapter entry=0,并可 reselect/wait/fail;ModelPreparationRejected无 Effect/Invocation,ModelDispatchAuthorizationRejected保留 not-executed Effect/Invocation/settlement; 两者均 ToolFeedback=0,只能关闭 Intent 后 wait/reject。只有新 committed resolution 才能以新 Intent 走两步续回合;- Runtime retry exhausted 才生成
RETRY_EXHAUSTEDToolFeedback;Adapter retryable 中间态不得驱动 selector 自行重试; - memory card disposition coverage、mapping、Recall Decision Outcome draft/link;
- DecisionMemoryBinding 的 NOT_REQUESTED/null/zero-card 与 EMPTY|DEGRADED|OK/snapshot/context 组合 validator;blocked recall 不进入 model;
- NOT_REQUESTED 只接受 actual Intent 中 admission-proof-backed
REGISTERED_NOT_REQUESTED;REQUIREDIntent/profile-proof tamper 必须 fail closed; - evaluator-private label/answer type 或 bytes 进入 request 时 fail closed。
- blocked diagnostic hint 只有 bounded summary/reason/remediation,不能 Inspect/ Execute/fallback;hidden deny 不能进入任何 actor bytes。
同一个 contract suite 至少运行 static-parity、hybrid-trajectory 和 deterministic fake policy artifacts;不通过 Python policy subclass 选择行为。
同一套测试运行 legacy/skill/in-memory source Adapter:
- canonical sort/round-trip/digest;
- duplicate identical fragment 可幂等合并;
- same ID/different schema、binding、risk、description fail closed;
- callable/typed request schema 与 manifest expected digest 一致;
- required config/auth 只存名称/scope,不泄露值;
- malicious
authorized: true或 unknown field 被拒绝; - source import failure 显式失败,不吞异常;
- catalog artifact 可在新进程重载并得到同一 digest;
- diff 输出新增/删除/行为变化/非行为 metadata 变化分类。
- stage/frozen/auth/config/environment/data-flow/risk/budget filter 顺序;
- descriptor/Selection/Execution policy 的 union/intersection/min/deny precedence 冲突矩阵,missing applicable allow 默认 deny;selection/final authorization 共用 normalized-policy digest/reason order;
- 被过滤 candidate 无论 score 多高都不能暴露;
- disposition 三分法和两 lane rank:executable ranker 永不看到 hidden/diagnostic, diagnostic ranker 永不产生 schema/alias/Effect 或 promotion;
- small-set bypass;
- absolute/context-fraction schema budget 取更严格值;
- lexical exact name/parameter match;
- semantic paraphrase dense match;
- deterministic fusion/tie-break;
- dense index 与 exact catalog/ordered descriptor corpus coverage;stale index、 catalog swap、partial index fail/degrade closed;
- pinned numeric runtime 的 clean-process/cross-worker query feature、fixed-point score、ranked IDs 和 directive byte golden;不匹配 worker fingerprint 拒绝 admission;
- hard negative 与 confused-with;
- adaptive top-k 与 required multi-tool set;
- dense unavailable 的每个显式 degradation;
- low-confidence expand/clarify/static-stage fallback;
- fallback 不能越过 original stage allowlist;
- ToolFeedback/subgoal/evidence/availability change re-selection;
- input map order、unrelated stats 不影响 active decision digest。
- Selection Bundle 的 tokenizer/analyzer/model/index/fusion/threshold/renderer 任一 行为 digest 变化均改变正确 identity;
- narrowing override 只接受预冻结 development artifact,含 evaluator case/label 字段即拒绝。
- classification join 对 input/recall/feedback/artifact 保守闭包;model argument claim
只能 add;伪造/缺失 declassification receipt、summary/truncation laundering、
EVALUATOR_PRIVATE|CREDENTIAL_SECRETegress 全部 fail closed;model 与 tool Effect 的 prepare/dispatch data-flow digest mutation matrix 完整。
Native 与 prompt-emulation Adapter 运行相同 cases:
- DIRECT no-tool;
- CLARIFY control;
- DISCOVER control;
- single EXECUTE proposal;multiple/parallel proposal 按 v1 contract 整体拒绝且零 Effect;
- COMPLETE/HANDOFF;
- malformed/double-encoded JSON;
- unknown provider alias;
- missing/wrong turn-contract digest;
- provider returns text plus tool call;
- provider returns parallel dependent calls;
- required/auto/none capability matrix;
- provider-specific downgrade 显式进入 profile digest;
- canonical proposal round-trip 与 wire order。
不能正确表达 optional no-tool 的 provider profile 不得进入 dynamic-auto-v1。
对每个 active Agent 和 provider profile保存 golden fixture,比较 legacy/shadow:
- model request bytes/digest;
- schema order/tokens;
- tool choice/parallel;
- number of provider calls;
- actual tool/control sequence;
- context patch/handoff/stage output;
- active cache identity/hit;
- active Effect/receipt;
- exception 与 termination behavior。
另外注入 shadow catalog corruption、ranker error、timeout 和 oversized trace,证明 active bytes、Run event sequence、active Budget Envelope 不变且 shadow failure 有界 可见。测试还必须证明 active path 不写 mode-specific outbox event、不等待 audit projector;selection worker 在 request artifact 提交后运行,且 production shadow 不发 remote/model Effect。隔离资源池下 p50/p95 active latency 必须通过预注册 overhead SLO。projector 在 missed notification/crash/duplicate scan 下按 derived job ID eventually-complete 且无重复 logical record。完整反事实 replay 使用独立 Run ID、 budget、journal 与 cache namespace。
tests/test_runtime/test_tool_authorization.py 至少覆盖:
- highest-ranked forbidden capability;
- capability exposed 后 credential revoked;
- provider alias collision;
- schema/binding/policy/toolset digest swap;
- canonical args 在 selection 后被篡改;
- stale Run state/event seq/Activity generation/fence;
- Budget Envelope insufficient;
- destructive approval missing/expired/wrong principal;
- private/evaluator data -> open-world destination;
- idempotency/reconcile contract incompatible;
- preparation denial 不创建 Effect/Physical Invocation/reservation;
- preparation acceptance 在一个 transaction 创建 Effect、
DISPATCHABLEinvocation、 reservation,但 Adapter entry 仍为 0; - prepare/dispatch 之间分别 revoke credential、remove config、expire approval、change
availability/budget、open incident、advance state/generation/fence 或 tamper data-flow:
dispatch transaction 必须提交
CONFIRMED_NOT_EXECUTED、settle reservation、发 closed trigger,且 Adapter entry=0; - exact unchanged bindings 只有在原子
DISPATCHABLE -> DISPATCHEDcommit 后才允许 Adapter entry;commit/reply crash 由 invocation read-back reconcile; - model provider 与 semantic summarizer 的 confidential/restricted egress 使用同一 preparation/dispatch authorization,未经 commit 不发送 bytes;
- model Effect 必须能在其 terminal Recall Decision Outcome 尚不存在时 dispatch,但须 绑定 current Intent/actor slot/Recall Input/optional Context;伪造旧 Intent、错误 slot、 foreign Context 或预先存在 Outcome 均拒绝;
- memory-informed accepted tool invocation 在
MemoryToolDecisionLinked前不能 获得DISPATCHED,移除/篡改 link 会得到 dispatch rejection,link append/replay 幂等。
同一套 contract 运行 in-process、subprocess、Docker、Browser 和 scripted fault Adapter(不适用项必须显式声明):
- one
invoke= at most one physical dispatch; - no hidden retry;
- request/capability/schema/binding/args digest validation;
- timeout、cooperative cancel、TERM/KILL 与 owned resource cleanup;
- query/reconcile known/not-executed/ambiguous;
- late evidence inbox;
- stale Adapter/worker 不能选择 receipt 或 settle;
- full output artifact + bounded ToolFeedback;
- Browser session identity/state isolation;
- Docker container/process ownership labels;
- non-cooperative in-process tool 被 contract 拒绝并要求 process Adapter。
每个 case 至少 kill/crash 于:
- Tool Decision Record 构建前/后;
- model Effect prepared 前/后;
- model receipt committed 后、proposal compile 前;
- tool proposal validated 后、authorization 前;
- continuation spec commit 后、child Intent/Input/Recall commit 前,以及其 commit 后、
第二次
advance()前; - authorization 后、Recall Decision Outcome/link 前;
- Effect/
DISPATCHABLEPhysical Invocation/reservation transaction 后; - dispatch reauthorization transaction 前/commit 后、Adapter entry 前;
DISPATCHED后、external Seam 前;- provider return 后、receipt commit 前;
- receipt commit 后、artifact/ToolFeedback 前;
- ToolFeedback commit 后、trajectory continuation spec/child input 前;
- stage proposal 后、StageContinuation transition 前。
恢复断言:同一 logical Effect、完整 independent Physical Invocation history、至多一个 selected receipt、正确 worst-case settlement、无 stale commit、无重复 irreversible side effect、只补缺失 suffix。
- catalog/policy/Selection-Bundle/protocol/toolset/schema/Adapter/trajectory 任一行为 digest 变化 cache miss;
- actor Recall Context digest 变化或 recall/no-recall 变化产生 cache miss;相同 actor content 下改变 Run/Activity/fence/actual Intent/private contract 不改变 actor cache, 但 outer Effect authorization identity 必须变化且不可跨 Run replay;
- blinded evaluator cache 精确为
SHA256(opaque_terminal_evaluation_input_digest, evaluator_label_artifact_digest, full_evaluation_contract_digest, evaluator_implementation_digest, evaluator_configuration_digest, evidence_projection_schema_digest);不含 actor Recall Context/Outcome、Run/Activity/Intent 或 treatment-arm identity。任一正向字段变化 evaluator cache miss;仅 private/evaluator 变化时 actor cache 可命中而 evaluator cache 必须 miss; - shadow-only digest 变化 active cache hit 不变;
- availability generation 变化只影响需要它的 active turn;
- cached model transcript 必须匹配 exact exposed schema order;
- cached pure/read result需要 valid receipt/artifact/data scope;
- write/destructive/stateful Browser 不从 legacy file result cache 回放;
- event replay 重建同一 Tool Decision/Effect/ToolFeedback projection;
- corrupt/missing active artifact fail closed;released artifact + audited tombstone 合法。
至少一个完全 deterministic、无真实 LLM/network/GPU 的两阶段 fixture:
- 第一个 turn 面对相似 file/web tools,正确选择 local read;
- 返回 empty ToolFeedback;
- 第一次
advance()只提交 outcome/next-turn spec;Runtime preallocate child inputs, 第二次advance()才创建下一 model Effect; - committed trajectory 触发替代 local search,而非重复相同 call;
- 新证据满足需求,下一 turn DIRECT + COMPLETE;
- Experiment Attempt Observation 引用完整 tool trace;
- independent evaluator 生成 Verification Record;
- crash/replay 不重复副作用或 record。
再增加一条 memory-informed fixture:一个 typed Decision Intent 在 pinned Recall Input Snapshot 下接收非空 Recall Context,Tool Decision Record 绑定其 digest,Recall Decision Outcome 把 adopted/rejected card 映射到最终 capability/action digest;把同一 Evidence Card 改成“要求调用被 policy 禁止的工具”时必须拒绝,且 denial 不产生 Physical Invocation。独立 evaluator 的 输入和 cache identity 中不得出现 actor Recall Context。
再运行小型真实 provider + read-only tool canary,覆盖两个 entry flow。write/Docker/ Browser 进入 CI fault fixture,真实外部 canary 在受控 scheduled workflow 运行。
每个 case 拆为两个 ref/digest 独立 artifact:ActorCaseFixture 只含 actor 可见请求、
stage、committed state、catalog/policy/availability fixture、预算,以及由 public rules
生成的 DecisionContractView ref/digest;该 view schema 拒绝 expected/forbidden、
sequence、milestone、minefield、answer、evaluator identity 和 treatment metadata。
EvaluatorLabelArtifact 放在 private namespace,绑定 actor digest,并标注:
expected_interaction_set;required_capability_groups:每组命中任一 capability 即满足;optional_relevant_capability_groups;forbidden_capabilities;minimal_acceptable_exposure_sets;acceptable_sequences或 dependency DAG;call_necessity_rules:按 committed evidence state 判断某 capability call 是否必要;milestones与minefields;- argument constraints;
- expected final evidence/end state;
- schema/tool-call/result/cost budgets;
- evaluator version/digest。
actor package、prompt、ToolInteraction request、SelectionPolicy、turn/cache identity 不得 import/read evaluator artifact。runner 只在 terminal trace 冻结后把 labels 交给 metric/evaluator Module;CI 有 dependency 和 byte-leak tests。同一 ActorCaseFixture/ DecisionContractView 配不同 private labels、evaluator identity 或 treatment metadata, actor request、turn-contract 和 cache-key bytes 必须完全相同。
独立 evaluator cache 的唯一 v1 positive identity 是以下 canonical tuple 的 SHA-256:
opaque_terminal_evaluation_input_digest
evaluator_label_artifact_digest
full_evaluation_contract_digest
evaluator_implementation_digest
evaluator_configuration_digest
evidence_projection_schema_digest
runner 必须把六项和 resulting cache digest 写入 raw evaluation artifact/acceptance manifest。Run/Activity/Decision Intent、actor Recall Context/Outcome 和 treatment-arm identity 禁止作为独立字段。任一六项变化 mandatory miss;改变 private labels/ contract/evaluator 时 actor request/cache 必须保持命中条件不变,而 evaluator cache 必须失效。
核心集合指标:
required_set_recall = satisfied_required_groups / all_required_groups
exposure_precision = exposed_relevant_capabilities / all_exposed_capabilities
excess_exposure = exposed_capabilities - minimal_acceptable_exposure
unnecessary_call_rate = unnecessary_executed_calls / all_executed_calls
forbidden_call_rate = forbidden_executed_calls / all_executed_calls
trace_completeness = terminally_accounted_attempts / all_provider_proposals
MetricSpec-v1 冻结以下唯一计算规则:
- 无 required group 的 direct/clarify case,
required_set_recall=null并从该指标 macro denominator 排除;另由 direct/clarify accuracy 计分,不记成 recall 失败; exposure_precision:expected relevant 与 exposed 都空时为 1;exposed 空但 required 非空时为 0;否则按 required+optional relevant union 计算;excess_exposure=max(0, |exposed|-min_acceptable_size);miss 由 recall 单独惩罚;- unnecessary call 由 frozen
call_necessity_rules对每个 committed pre-call state 判定,不依赖事后模型主观打分; - forbidden-call 零调用分母记 0;trace-completeness 零 tool-proposal case 记 1;
- 先按 Research Run 算,再按 registered case/seed pair 聚合;主报告 task-family macro,micro 只作诊断;multi-tool 同时评 set 与顺序。
ReleaseCriteria-v1 是机器可读 artifact,包含 primary/secondary metrics、方向、
margin/threshold、family harm guardrail、样本量/power、alpha、多重性、paired CI
estimator/bootstrap cluster、stopping/missing/invalid-run rules、latency/cost bound 和
MetricSpec/analyzer digests。T0 baseline 后、adaptive assignment/result 解盲前冻结;
analyzer 拒绝 CLI 临时覆盖。
先做独立 migration-parity 实验:真实 legacy path 对 canonical static Adapter,保持 legacy provider/control bytes 一致,只证明迁移无行为变化;该实验不估计 selection 收益。
selection ablation 的五个 arm 全部使用同一个 canonical optional-tool/typed-control provider profile,只改变 SelectionPolicy/Bundle:
A CANONICAL_STATIC
B HARD_FILTER_STATIC
C DENSE_TOP_K
D HYBRID_ADAPTIVE
E HYBRID_TRAJECTORY
固定:
- model/provider/profile 与 sampling config;
- case、seed/arm order/randomization;
- Capability Catalog 与 concrete Tool Implementations;
- Evaluation Contract/evaluator;
- Budget Envelope、timeouts、output policy;
- prompt/history/input artifacts;
- environment/credential availability fixtures;
- cold/warm cache protocol;
- failure injection schedule。
每个 arm 使用不同 selection policy/Selection-Bundle digest,但不能看到其他 arm 的 cache、trace、ToolFeedback 或 derived statistics。arm assignment 与 analysis plan 在 正式运行前归档。
至少报告:
- no-tool/direct;
- clarification;
- small compact stage set;
- over-budget large set;
- single-tool;
- multi-tool dependency;
- hard-negative confusion;
- availability/auth;
- risky/egress;
- error recovery;
- long-output;
- prompt injection;
- provided-idea/reference-ideation flow;
- provider profile;
- catalog size/distractor similarity。
不得只报告总体平均数掩盖危险工具或 multi-tool recall 退化。
发布结论按以下顺序:
- trace/catalog/protocol contract 正确;
- safety/governance gate;
- final valid end-state noninferiority;
- selection diagnostics;
- schema/result token、latency 和 cost 改善;
- trajectory-aware 增益;
- 只有 verified trace 规模和离线/在线实验足够后,才讨论 learned reranker、 fine-tuning 或 RL。
不能用 relevance score 或 schema token reduction 替代 Verification pass。
Research Run spec pin:
tool_use:
profile_ref: artifact://tool-profiles/dynamic-auto-v1.json
profile_digest: "..."
catalog_ref: artifact://tool-catalogs/catalog-v1.json
catalog_digest: "..."
selection_policy_ref: artifact://tool-policies/adaptive-v1.yaml
selection_policy_digest: "..."
selection_bundle_ref: artifact://tool-selection/bundle-v1.json
selection_bundle_digest: "..."
execution_policy_ref: artifact://tool-policies/execution-v1.yaml
execution_policy_digest: "..."
provider_protocol_ref: artifact://tool-protocols/native-v1.json
provider_protocol_digest: "..."
adapter_bundle_ref: artifact://runtime-adapters/tool-v1.json
adapter_bundle_digest: "..."
normalized_tool_configuration_digest: "..."
release_criteria_ref: artifact://tool-evals/release-criteria-v1.json
release_criteria_digest: "..."
deployment_bundle_ref: artifact://deployments/tool-runtime-v1.json
deployment_bundle_digest: "..."full Evaluation Contract/Run/Activity/fence/actual Decision Intent identity 仍在 Runtime
spec/envelope,不进入 actor-visible Research Context Snapshot 或 actor cache。Snapshot
只带 sanitized DecisionContractView 与上述 normalized tool configuration 的 actor-safe
behavioral projection;private-only contract 变化创建新 Run,但不改变相同 actor content
的 transcript cache bytes。
进程环境变量只能选择新 Research Run 的已批准 bundle,不能改变 in-flight Run。
- all-stage shadow,至少覆盖预注册 case/run 数量;
- internal deterministic/read-only canary;
- 真实 provider read-only canary;
- planning/external-search canary;
- write/command/Docker/Browser risk canary;
- 10% new Research Runs;
- 50%;
- 100% default;
- 完成正式 A/B、rollback drill 后删除 legacy production path。
每档都需要覆盖 fault-free window 和预注册 task family,不以“运行了几天”替代样本 与事件数。
以下 integrity/security 事件是紧急隔离,不允许“in-flight 继续 pinned bundle”:
- forbidden/destructive/egress authorization bypass > 0;
- unknown/ambiguous outcome 被渲染为 success > 0;
- stale fence/toolset/schema/args commit > 0;
- duplicate irreversible side effect > 0;
- trace completeness < 100%;
- cache cross-policy/cross-arm leakage > 0;
- catalog integrity/replay failure;
- ToolFeedback redaction/private-data canary 泄漏。
同一个 authenticated system-control transaction 关闭受影响 admission、建立 incident
epoch/scope、撤销受影响 worker fence 和尚未 DISPATCHED 的授权。受影响 Run 进入
durable design 规定的 quarantine:WAITING_INPUT、cancel/terminate/reconcile 或
typed failed disposition;incident scope 内新 DISPATCHED=0。只有确认不在 scope 的
健康 Run 才继续。
以下 scientific/performance regression 只停止新 adaptive admission:final noninferiority 失败、p95 latency/cost 越界。健康 in-flight Run 保持其 pinned bundle, 避免中途改变实验条件。Provider malformed/unavailable 按 typed availability policy 进入 refresh/wait;若同时构成完整性风险则升级为 incident。
每个 deployment bundle 绑定 code revision、journal/event schema readers、upcasters、
canonical serializer、provider/Tool Adapters 和 compatibility matrix。发布至少保留
current、previous durable bundle,以及任何仍被 in-flight Run 引用的 bundle;只有
引用计数为 0、settlement 完成、replay/retention gate 通过后才 GC。Append-only event
不做 downgrade rewrite;旧 binary 无法解析新 schema 时停止 admission/进入
WAITING_INPUT,不能丢字段继续。删除 legacy 后,回滚目标只能是 retained previous
durable bundle,禁止恢复 mutable Registry/direct callable authority 来“救火”。
要求
trace_completeness == 1.0;- 每个 proposal/attempt 有唯一 terminal disposition;
- error/denial/empty/ambiguous 分类正确;
- raw trace 能重算 tokens/latency/cost/metrics。
证据
pytest -q tests/test_core_usage.py tests/test_workflow/test_flowcache.py tests/test_tool_use/test_shadow_inertness.py- baseline manifest、raw trace 和 analyzer digest。
要求
- active capability 100% canonicalized;
- collision/schema/binding drift 0;
- no secret/live handle in catalog;
- static policy 可重建每个 baseline Agent set;
ToolConfigurationValidator产生并验证 normalized catalog/policy/bundle/protocol/ Adapter configuration digest;Catalog compiler 不拥有 cross-policy authority。
证据
python -m research_agent.inno.tool_use.catalog checkpytest -q tests/test_tool_use/test_catalog_contract.py- catalog artifact、inventory 和 diff report。
要求
- pure determinism;
- bounded canonical views;Selection Bundle 完整绑定;
- closed interaction/control semantics;
- v1 one-tool-call-per-turn、preparation/dispatch-rejection closed 回流;
- model-vs-tool rejection trigger/lineage/continuation matrix 全覆盖,model rejection 永不伪造 ToolFeedback;
- next-turn continuation 必须分成 outcome/input commit 与后续 prepare 两次调用;
- executable/diagnostic 两 lane 均先 policy classification 再独立 rank,diagnostic 永不暴露 schema/alias 或执行;
- schema budget;
- evaluator label/input/cache isolation;
- all enabled provider profiles pass optional no-tool/control round-trip。
证据
pytest -q tests/test_tool_use/test_interaction_contract.py tests/test_tool_use/test_policy.py tests/test_tool_use/test_protocol_contract.py- provider compatibility matrix。
要求
- active bytes/digests/effects/results 与 legacy一致;
- shadow failure 不影响 active lane;
- shadow 不进入 active cache、Run event sequence、Budget Envelope 或 dispatch;
- production shadow 只从 non-causal outbox selection-only 执行,无 remote/model call,且 active path 不等待;
- 隔离 worker pool 下的 p50/p95 active-latency overhead 通过预注册 SLO。
证据
- all-Agent golden report;
- both-flow shadow run artifacts;
- shadow fault injection report。
要求
- forbidden exposure/call、authorization bypass、sensitive egress、approval bypass、 stale commit、alias/schema/args swap 均为 0;
- model/tool preparation denial logical Effect/Physical Invocation = 0;tool dispatch rejection 保留
已有 lineage、
CONFIRMED_NOT_EXECUTED/settlement 完整且 Adapter entry=0; - model/tool/summarizer egress 的 provenance/classification/declassification 重算无 bypass;
- prompt injection minefield触发 = 0;
- malicious Evidence Card/Recall Context 不得扩大 eligibility、approval、scope 或 egress;对应 denial Physical Invocation = 0;
- memory-informed tool invocation 在 Recall Decision Outcome link 前
DISPATCHED=0; - blocked diagnostic 的 capability identity、summary、reason、remediation 四组逐字段 leak matrix 全过;visibility 无匹配/冲突/deny 时 actor bytes 均为 0。
证据
pytest -q tests/test_runtime/test_tool_authorization.py tests/test_tool_use/test_policy.py- security case Evaluation Contract report。
要求
- one retry owner;
- every dispatch has Effect/invocation/reservation;
DISPATCHABLE -> DISPATCHED是唯一 Adapter-entry authorization;prepare/dispatch 间 mutable-state race 全部 closed not-executed;- receipt/reconcile/ambiguity/settlement 正确;
- continuation input/outcome/child Intent 与 Effect/dispatch fault seams 只补 missing suffix;
- Adapter 无 owned-resource leak。
证据
pytest -q tests/test_runtime/test_tool_effect_adapters.py tests/test_runtime/test_tool_effect_faults.py- scripted fault raw events、projection replay 和 budget report。
要求
- 所有行为 digest 变化正确 miss;
- Selection Bundle 任一行为 asset 变化正确 miss;
- shadow-only 变化不污染 active;
- cross-arm/cross-policy leakage = 0;
- actor recall/no-recall 正确隔离,blinded evaluator identity 不泄露 recall/arm;
- 同一 actor fixture/DecisionContractView、不同 evaluator-private labels/contract digest 得到逐字节相同 actor request、turn contract 与 cache key;
- 同 actor content 下 Run/Activity/fence/actual Intent/private contract 变化不进入 actor cache,但 Runtime envelope/outer Effect identity 改变且 cross-Run Effect reuse=0;
- evaluator cache 精确绑定 terminal opaque input、labels、full contract、evaluator implementation/configuration 和 evidence projection schema;六项任一变化 miss;
- replay 产生相同 Tool Decision/Effect/ToolFeedback projection。
- Tool Decision Record 保持纯因果边界,later events 只在 projection join。
证据
pytest -q tests/test_tool_use/test_cache_identity.py tests/test_workflow/test_flowcache.py- replay/invalidation matrix artifact。
要求
- paired Run-level valid-pass delta 的 95% CI 下界高于预注册 margin;默认 margin
不得宽于
-2percentage points; - required-capability-set recall 和 direct/clarify macro accuracy 不低于预注册 baseline bound;
- over-budget stratum 的 exposed schema tokens 满足 policy 且 paired reduction > 0;
- small-set bypass 不增加 selector model/remote Effect;
- runtime/selection overhead 的 median/p95 bound 在 T0 后、正式结果前冻结;
- migration parity 与 common-profile selection ablation 分开,不混入 protocol/control 差异;
ReleaseCriteria-v1、MetricSpec、样本量/power/CI estimator 在解盲前可验证冻结;- 任意 efficiency claim 同时报告 end-state、调用数、tokens、latency、cost 和方差。
证据
python -m benchmark.tool_use.run --manifest ...python -m benchmark.tool_use.analyze --manifest ...- preregistered analysis plan、randomization、raw results、power simulation 和 Verification Records。
要求
- 两个 entry flow 和全部 production Agent 通过目标路径;
- direct callable/control-as-tool/MD5/global overwrite/duplicate cache-retry-trace authority production caller = 0;
- rollback 不依赖 deleted legacy path;
- current/previous/in-flight deployment bundles 可重放;integrity incident 能关闭
admission、revoke fence、quarantine/reconcile,受影响 scope 新
DISPATCHED=0; - docs/index/status 与当前代码一致。
证据
- dependency/import tests;
rgdeletion manifest;- full test suite;
- rollout/rollback drill;
- acceptance manifest。
每个 release candidate 归档机器可读 manifest,至少包含:
git_revision
python_dependency_os_provider_fingerprint
tool_interaction_schema_version_and_digest
capability_catalog_ref_and_digest
catalog_inventory_and_conflict_report_digest
selection_and_execution_policy_refs_and_digests
selection_bundle_ref_and_digest
provider_protocol_matrix_and_adapter_bundle_digest
normalized_tool_configuration_digest
deployment_bundle_and_schema_compatibility_matrix_digest
research_context_snapshot_and_cache_identity_schema_digest
actor_scope_runtime_authorization_envelope_linkage_digest
baseline_manifest_and_raw_trace_digest
shadow_golden_and_fault_report_digest
actor_case_corpus_and_private_evaluator_label_digests
evaluator_cache_identity_and_invalidation_report_digest
metric_spec_and_release_criteria_refs_and_digests
arm_randomization_and_preregistered_analysis_digest
model_tool_environment_budget_fingerprints
tool_decision_and_effect_trace_digest
memory_decision_outcome_binding_report_digest
authorization_security_report_digest
data_flow_provenance_declassification_report_digest
dispatch_reauthorization_and_not_executed_report_digest
adapter_contract_and_fault_matrix_digest
cache_replay_invalidation_report_digest
raw_usage_latency_cost_and_end_state_results_digest
noninferiority_and_efficiency_analysis_digest
rollout_stage_task_counts_and_incidents
rollback_drill_digest
legacy_deletion_and_dependency_scan_digest
known_waiting_input_and_ambiguous_effect_cases
每个 digest 必须解析到随 release candidate 保存的 immutable artifact;只有 hash 没有可取回内容不算证据。正式分析前归档 raw results、arm assignment 和 analysis plan。手工删除失败 case、重分类 error、替换 catalog/tool output 或修改 evaluator 必须写 manifest deviation,并使对应 gate 失败或重跑。
建议 PR 顺序:
- Trace truth: internal attempt trace、typed disposition、baseline capture。
- Canonical contracts: serializer、deep-immutable values、supporting policy types。
- Catalog contracts: canonical descriptors、legacy/skill Adapters、inventory。
- Selection assets: bounded views、Selection Bundle、static policy、renderer。
- ToolInteraction core: pure Interface、single-call protocol、feedback/rejection。
- Evaluation harness: actor/private-label split、MetricSpec、runner/analyzer。
- Shadow Adapter: non-causal outbox worker、MetaChain bridge、causal/budget inertness 与 active-latency overhead SLO。
- Typed control: provider protocol、completion/wait/handoff、read-only canary。
- Hybrid selection: lexical+dense/fusion、progressive disclosure、hard negatives。
- Trajectory: committed feedback/subgoal/availability re-selection。
- Runtime authorization: sealed transaction、Effect request、reservation。
- Memory linkage: Recall Decision Outcome guard 与 crash/replay。
- Execution Adapters: in-process/subprocess/Docker/Browser + fault suite。
- Typed output: artifact/bounded feedback、legacy tool migrations。
- All-stage migration: policies、两个 entry flow、control tool removal。
- Formal evaluation and rollout: parity/ablation、fault、canary、incident/rollback。
- Deletion: old production authorities、migration Adapter、stale docs/tests。
不要把 1--15 合成一次大改。每个 PR 必须有可独立 review 的 Interface-level contract test 和明确 rollback point。
- Phase 0 baseline manifest 与 raw traces 在行为修改前归档;
- active capability universe 100% 进入 canonical Capability Catalog;
- catalog 无 collision/schema/binding drift、secret 或 silent import failure;
- ToolInteraction 只有一个 pure
advance()Interface,不读 artifact/clock/remote dependency,并在 pinned Selection Bundle 下通过 determinism suite; - DIRECT/CLARIFY/DISCOVER/EXECUTE 与 COMPLETE/HANDOFF typed 且不混入 Tool Effect;
- executable 与 policy-approved blocked-diagnostic 两 lane 都先完成 closed policy classification 再独立 ranking;hidden deny 不进入 actor bytes,diagnostic 永不进入 Inspect/Execute/fallback;diagnostic visibility/remediation 来自 content-digested ExecutionPolicy,no-match/conflict/deny fail closed,逐字段 private-capability leak=0;
- descriptor/Selection/Execution 通过一个 fail-closed policy algebra 合成, selection 与 final authorization 共用 normalizer/reason precedence;
- small-set bypass、schema budget、internal Catalog/Inspect/Execute views 和 explicit degradation 可验证,且不读取 evaluator labels;
- MD5 或其他无语义的 silent fallback 已删除;
- dense index 绑定 exact catalog/descriptor corpus 和 deterministic numeric runtime; stale/partial/mismatched worker 被拒绝或显式降级,cross-worker byte golden 通过;
- 每次
advance()恰有一个 causal-pure Tool Decision Record;每个 model turn 有 exact toolset/turn-contract digest,later events 不回写 record; - actor
ToolInteractionRequest只含 ActorTurnScope;Run/Activity/fence/full private contract/actual Intent 只在 RuntimeAuthorizationEnvelope,journal 双层 binding 可 read-back,actor cache byte-isolated 且 Effect 不可跨 Run replay; - memory-informed model decision 绑定 Decision Intent、Recall Input Snapshot 与
optional Recall Context;model Effect 在 Outcome 前可 dispatch,settled
EXECUTEproposal 的 tool Effect 另绑定 terminal Recall Decision Outcome;Evidence Card 不能 改变 eligibility/approval/authorization,tool outcome link 前新DISPATCHED=0; - NOT_REQUESTED actor binding 只能由 actual Intent 的 verified
REGISTERED_NOT_REQUESTEDgovernance binding 投影;REQUIRED Intent 无降级通道; - 每个 memory-governed model turn 使用新 Intent;child turn 删除旧 recall bytes、 绑定 parent outcome,并按新 committed state 重新 recall/记录 not-requested;
- DISCOVER/feedback/input/protocol/tool preparation/tool dispatch rejection 需要续回合时的第一次
advance()只提交旧 outcome 与 NextTurnInputSpec;Runtime preallocate child Intent/Input/Recall 后第二次调用才可准备 model Effect,crash seams 不重复; - shadow 通过 non-causal outbox 脱离 active critical path,active bytes/Run event sequence/Budget byte-inert,满足 overhead SLO,并覆盖两个 flow/全部 Agent;
- enabled provider profile 支持正确 optional no-tool 和 control protocol;
- 每个 external tool dispatch 是 Runtime Activity 下的 Effect/Physical Invocation;
- preparation transaction 只创建 Effect/
DISPATCHABLEinvocation/reservation;紧邻 Adapter entry 的原子 dispatch transaction 重验全部 mutable/immutable authority, 成功后才DISPATCHED; - v1 每个 model turn 最多一个 external tool call;tool preparation rejection 回流且 Effect/Invocation=0;
- tool dispatch rejection 保留 Effect/Invocation,提交
CONFIRMED_NOT_EXECUTED、settle reservation、Adapter entry=0,并经 closed trigger 回流; - model preparation/dispatch rejection 分别保持 zero-Effect 与 existing-not-executed lineage,ToolFeedback=0,关闭 Intent 后 wait/reject;新 model turn 只能来自新状态;
- retriever/ranker 无路径调用 execution Adapter;
- one retry owner、late receipt、ambiguity、reconcile、settlement contract 完整;
- in-process/subprocess/Docker/Browser/fault Adapter contract suites 全过;
- ToolFeedback typed、bounded、artifact-backed,
LEGACY_UNTYPED为 0; - tool preparation rejection 四态与 Effect terminal/retry-exhausted status 闭合; selector 不拥有 retry;
- Decision Input/Recall/arguments/model requests/tool output/summaries 全部带 closed classification/provenance;conservative join、trusted declassification、model/tool prepare+dispatch egress tests 全过且 private/credential 泄漏为 0;
- ToolFeedback 与 Experiment Attempt Observation 的身份和命名分离;
- 每个 committed feedback/failure/evidence/subgoal/availability change 可触发重选;
- dependent calls 跨 committed ToolFeedback turns 串行正确;v1 未因 provider parallel flag 绕过 Runtime;
- Research Context Snapshot 包含 catalog/policy/Selection-Bundle/protocol/Adapter digests 和 actor-visible DecisionContractView;actor-turn cache 另绑定 decision/recall/trajectory identity,且不绑定 evaluator-private contract digest;
- cross-arm/cross-policy/cache leakage 为 0;
- tool-use Evaluation Contract 覆盖 no-tool、clarify、hard negative、multi-tool、 auth/risk/error/long-output/injection/trajectory;
- trace completeness 100%,safety/authorization/stale/duplicate violations 为 0;
- independent evaluator 不接收 actor Recall Context,evaluation cache 也不以其为 输入;
- evaluator cache 正向绑定 opaque terminal input、private labels、full contract、 evaluator implementation/configuration 与 evidence projection schema;任一变化 miss,而 private-only 变化不扰动 actor cache;
- actor case 与 evaluator-private labels 物理隔离;MetricSpec/ReleaseCriteria、 power/CI/threshold 在 adaptive 解盲前冻结;
- end-state noninferiority、selection、schema budget、latency/cost gates 全过;
- provided-idea 和 reference-ideation flow 都使用共同目标路径;
-
case_*/transfer_*不再是 Tool Capabilities; - direct callable dispatch、global overwrite/eager import、public旧 search、重复 ToolModule authority 与 migration session 已从 production 删除;
- rollout、integrity incident quarantine、replay-compatible durable rollback drill、 acceptance manifest 和 raw artifacts 完成;
- governing design/index/README 状态与实际代码一致。
本轮交付不要求:
- MCP registry/server marketplace、A2A 或跨组织 tool sharing;
- remote learned router、fine-tuning、RL;
- distributed/parallel tool DAG;
- schema compression;
- active Research Run 热安装新工具;
- 把 Runtime recovery 或 token reduction 描述成 research intelligence gain。
只有 T8 完成且 verified traces 足够后,后续 policy version 才可评估:remote reranker、 learned need-tool gate、tool dependency graph learning、并发 wave、programmatic calling 和 schema compression。每一项都必须作为独立 Intervention,在相同 Evaluation Contract/Budget Envelope 下产生 Observation 与 Verification Record。