File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -119,6 +119,21 @@ control rather than hidden implementation details.
119119 <em >Ablation-derived component contributions across task success, API-cost saving, and latency saving.</em >
120120</p >
121121
122+ ## Tau3 benchmark results
123+
124+ The latest three-way comparison was run against Hypha ` main@ef33263 ` with one trial per task
125+ (385 tasks total). The base model was ` deepseek-v4-flash ` . Hypha is the framework result and
126+ Direct is the direct model-call baseline. Neither arm reported a system error.
127+
128+ | Benchmark | Tasks | Hypha | Direct |
129+ | --------- | ----: | ----: | ------: |
130+ | Mock | 10 | 0.800 (8/10) | 0.800 (8/10) |
131+ | Airline | 50 | 0.880 (44/50) | 0.900 (45/50) |
132+ | Retail | 114 | 0.912 (104/114) | 0.895 (102/114) |
133+ | Telecom | 114 | 0.605 (69/114) | 0.588 (67/114) |
134+ | Banking knowledge | 97 | 0.206 (20/97) | 0.196 (19/97) |
135+ | ** Total** | ** 385** | ** 0.636 (245/385)** | ** 0.626 (241/385)** |
136+
122137## Product model
123138
124139| Concept | Responsibility |
Original file line number Diff line number Diff line change @@ -114,6 +114,20 @@ Invalidation 与 Bypass 都属于 Runtime 控制,而不是隐藏的实现细
114114 <em >通过 Ablation 得出的组件贡献,覆盖任务成功率、API 成本节省与延迟节省。</em >
115115</p >
116116
117+ ## Tau3 Benchmark 结果
118+
119+ 最新三方对比基于 Hypha ` main@ef33263 ` ,每个任务运行一次,共 385 个任务。Base model 为
120+ ` deepseek-v4-flash ` 。Hypha 列为框架结果,Direct 列为直接模型调用基线;两组均未报告系统错误。
121+
122+ | Benchmark | 任务数 | Hypha | Direct |
123+ | --------- | ----: | ----: | ------: |
124+ | Mock | 10 | 0.800(8/10) | 0.800(8/10) |
125+ | Airline | 50 | 0.880(44/50) | 0.900(45/50) |
126+ | Retail | 114 | 0.912(104/114) | 0.895(102/114) |
127+ | Telecom | 114 | 0.605(69/114) | 0.588(67/114) |
128+ | Banking knowledge | 97 | 0.206(20/97) | 0.196(19/97) |
129+ | ** 合计** | ** 385** | ** 0.636(245/385)** | ** 0.626(241/385)** |
130+
117131## 产品模型
118132
119133| 概念 | 职责 |
You can’t perform that action at this time.
0 commit comments