[functionstackx]

Add a bounded B300 NVFP4 HiCache experiment stacked directly on #4711 (6950741). The passed parent sweep still gains throughput at concurrency 32, but its low KV-pool occupancy, high actual cache hit rate, and lack of observed retractions do not establish that CPU caching will improve the frontier. This child tests that hypothesis without changing the parent configuration.

The new performance key pairs cache OFF and ON at concurrency 32, 48, 64, and 96. Both arms use max-running-requests=96 and cuda-graph-max-bs-decode=96, compared with 64/64 in the parent; compare the child arms directly rather than attributing the cap change to HiCache. OFF uses kv-offloading=none; ON uses kv-offloading=dram with backend identity hicache, enable-hierarchical-cache, write_through, kernel, and page_first.

ON uses only hicache-host-memory-fraction=0.65 for native automatic host allocation. It supplies no explicit hicache-size or hicache-ratio, no custom memory guard, and no engine patch. Actual allocation depends on live available RAM/cgroup headroom, local rank count, and the upstream combined target/Mamba/draft/indexer budget; the configured fraction is not an observed pool size.

The required ON dram-utilization: 1.0 metadata means 100% of the existing nominal eligible DRAM budget, subject to the generator's legacy approximately 3,000-GB cap; it is neither allocated/reserved host RAM nor the native HiCache fraction. OFF's generated zero denotes no offload metadata, not zero physical RAM. Both arms retain the same eight-GPU exclusive Slurm node and physical host allocation; fresh runtime evidence must report actual pool bytes separately.

The experiment keeps the official digest-pinned SGLang v0.5.22-cu130 image and build, checkpoint revision, TP8/EP1, NVFP4 target, shipped draft-loading path, disabled precision-conversion flags, EAGLE 5/1/6, full 1,048,576-token context, unfiltered 393-entry AgentX corpus, and 3,600-second duration. Performance uses the shared connector's measured AL 3.603536; it does not collect another curve.

Two append-only changelog entries separate performance from accuracy: the paired performance key has no-evals: true, while a dedicated HiCache-ON c32 key has evals-only: true and all-evals: true. This preserves one explicit ON canonical GSM8K evaluation with all 1,319 examples and synthetic acceptance disabled, rather than allowing OFF/ON engine-environment grouping to drop the ON evaluation. The existing OFF accuracy evidence belongs to the parent; it is not a new child result.

This is an experimental draft with no GPU sweep label. Before any full child sweep, review live host headroom and resolved pool bytes, then establish a real host-eviction/restore correctness fixture with synthetic acceptance disabled. Merely starting an ON server or passing a low-concurrency canary does not prove host restoration. The parent's empty-stream errors remain unresolved; no child throughput gain, error-free behavior, or live draft-tensor dtype observation is claimed.

中文

本实验直接叠加在 #4711 的 6950741 上，不修改父 PR 配置。父分支的成功 sweep 在并发 32 仍有吞吐增长，但 KV 占用较低、实际 cache hit 较高，且未观察到请求回退，尚不能证明 CPU HiCache 会改善前沿，因此新增受控 OFF/ON 对照。

两组均使用并发 32、48、64、96，统一 max-running-requests=96 和 cuda-graph-max-bs-decode=96；父配置为 64/64，比较时需区分这一容量变化。OFF 标识为 kv-offloading=none；ON 标识为 kv-offloading=dram、backend hicache，并启用 write_through、kernel、page_first。ON 仅设置原生自动预算 hicache-host-memory-fraction=0.65，不指定 size/ratio，不添加自定义内存 guard 或引擎补丁；实际 pool 大小必须由运行时可用内存、cgroup、rank 数和联合预算确认。

ON 所需的 dram-utilization: 1.0 表示使用现有名义可用 DRAM 预算的 100%，仍受生成器约 3000 GB 的历史上限约束；它不是实际分配或预留的 RAM，也不是 HiCache 原生 fraction。OFF 生成的零值表示没有 offload 预算元数据，不代表没有物理 RAM。两组保留相同的八 GPU 独占 Slurm 节点和实际 host 资源，后续需单独记录实际 pool 字节数。

保留官方 digest 固定镜像、checkpoint revision、TP8/EP1、NVFP4 target、checkpoint 原始 draft 加载路径、禁用精度转换标志、EAGLE 5/1/6、完整 1,048,576-token 上下文、未过滤的 393-entry AgentX 数据集和 3600 秒时长。性能测试通过共享连接器使用实测 AL 3.603536，不重新采集曲线。

changelog 将性能与准确率分开：八个性能点使用 no-evals: true，单独的 HiCache-ON c32 key 使用 evals-only: true 和 all-evals: true，保留完整 1319 题的 canonical GSM8K，并关闭 synthetic acceptance，避免 OFF/ON 合并时漏掉 ON eval。已有 OFF 准确率结果来自父 PR，不是子 PR 的新结果。

本 PR 保持实验性 draft，未添加 GPU sweep 标签。全量运行前必须审查实际 host headroom、pool 分配，并在关闭 synthetic acceptance 的条件下验证真实 host eviction/restore；成功启动或低并发 canary 不等于完成 restore 验证。父分支的空内容响应错误仍未解决，本 PR 不宣称已取得吞吐提升、零错误或直接观测到运行中 draft tensor dtype。

AI model disclosure

GPT-6 Astra coordinated the experiment and implemented this B300 NVFP4 child. Delegated GPT-6 Astra agents audited exact-build HiCache support and host budgeting, the dedicated evaluation plan, and app metric definitions.

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=41352674113

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=41352674113

[Oseltamivir]

/use 41352674113

[github-actions]

@Oseltamivir staging run 41352674113. Existing staged runs will be preserved; a completion link will be posted here.

[Oseltamivir]

Superseded by #4710 (merged as 23a74c6), which consolidates GLM-5.3-Flash B300 FP8 + NVFP4 base and HiCache points into one PR (no TP4 on the 8-GPU node). / 已由 #4710（合并为 23a74c6）取代，该 PR 将 GLM-5.3-Flash B300 FP8 与 NVFP4 的基础点和 HiCache 点合并为一个 PR（8 卡节点不跑 TP4）。