[functionstackx]

Test GLM-5.3-Flash NVFP4 at TP4 on B300, stacked directly on the unchanged TP8 HiCache draft #4729 (b4acc6d). The new performance matrix has an OFF concurrency-8 anchor and paired OFF/ON points at 32, 48, 64, and 96. Both TP4 arms keep the TP8 child's admission and decode-graph caps of 96, so this experiment does not add a cap change to the TP comparison.

The recipe uses one TP4/EP1 worker, four active GPUs, and the existing exclusive eight-GPU node reservation. It does not launch a second replica or release the other four GPUs. Report both frontiers: active-GPU-normalized throughput/cost and the full reserved-node throughput/cost. Under the same per-GPU hourly-price assumption, total tokens per dollar charged against eight reserved GPUs are half the value charged against four active GPUs, while cost per token doubles; an active-GPU improvement is not automatically a node-level economic improvement. Verify actual allocation with Slurm AllocTRES, worker visibility, and the resource snapshot's configured/backend GPU counts.

The same pinned image's GB300 TP4 parent sweep already exercised TP4/EP1, EAGLE 5/1/6, TRT-LLM DSA, FP8 KV, and FlashInfer TRT-LLM MoE successfully. Its runtime reports about 48.71 GiB/rank of model-plus-NextN loading memory and a 221.49-GiB combined subtotal of those allocations, persistent KDA, speculative scratch, and target/draft KV at cap128. Scaling only the per-request speculative scratch from 129 to 97 slots yields approximately 214.70 GiB at cap96, below B300's approximately 228.30-GiB static budget at fraction 0.85. These are planning estimates from an ARM64 GB300 run, not measured x86 B300 TP4 pool sizes or a runtime pass; auto-sizing, CUDA graphs, and actual headroom still need verification.

ON uses native hicache-host-memory-fraction=0.65, write_through, kernel, and page_first, with no explicit size/ratio, custom memory guard, or engine patch. dram-utilization: 1.0 is nominal metadata under the existing generator cap and four-of-eight GPU share (1,499 GB), not an enforced reservation or measured pool size. The native allocator may see the full exclusive node's live host memory; record its resolved combined target/Mamba/draft/indexer pools and cgroup limits.

Preserve the parent's official digest-pinned SGLang v0.5.22-cu130 image, model revision, NVFP4 target and shipped draft-loading path, disabled conversion flags, EAGLE 5/1/6, full 1,048,576-token context, unfiltered 393-entry AgentX corpus, and 3,600-second duration. Performance uses the existing shared measured AL 3.603536, with no new collection. The existing TP8 configurations and ON c32 evaluation remain unchanged.

TP4 receives fresh canonical accuracy checks for both arms: OFF c16 and ON c32, each using the full 1,319-example GSM8K suite with synthetic acceptance disabled. Separate evaluation keys plus evals-only/all-evals retain both rows, and different evaluation concurrencies avoid the existing evaluation-identity collision. The nine performance rows use no-evals.

This remains an experimental draft without GPU sweep labels. Before any full sweep, inspect actual allocations and pass the real host-eviction/restore correctness fixture with synthetic acceptance disabled; server startup alone is not restore proof. Existing empty-stream errors remain unresolved, and neither a TP4 B300 runtime pass nor an improved frontier is claimed.

中文

本 PR 直接叠加在未修改的 TP8 HiCache draft #4729（b4acc6d）上，测试 B300 的 GLM-5.3-Flash NVFP4 TP4。性能矩阵包含 OFF 并发 8 锚点，以及 32、48、64、96 的 OFF/ON 配对。两组均保留 TP8 子分支的 admission/decode-graph cap96，不额外改变 cap。

配置只启动一个 TP4/EP1 worker，使用四张 GPU，但仍保留原有八 GPU 独占节点，不启动第二个副本，也不释放另外四张卡。必须同时报告按四张活跃 GPU 归一化的前沿和按八张预留 GPU 计费的整节点前沿；在相同每 GPU 小时价格假设下，按八张预留 GPU 计算的每美元总 token 数为按四张活跃 GPU 计算值的一半，每 token 成本则翻倍。活跃 GPU 效率提高不代表整节点经济性一定提高，需检查 AllocTRES、worker 可见设备和 resource snapshot 的 configured/backend GPU 数。

同一固定镜像在 GB300 上已有 TP4/EP1、EAGLE 5/1/6、TRT-LLM DSA、FP8 KV、FlashInfer TRT-LLM MoE 的成功运行。根据其 cap128 实测分项，仅将 speculative scratch 从 129 个槽位换算为 97 个槽位，cap96 计划占用约 214.70 GiB/rank，低于 B300 fraction0.85 对应的约 228.30 GiB 静态预算。这只是 ARM64 GB300 的跨硬件估算，不是 x86 B300 TP4 实测；实际 pool、图捕获和剩余内存仍需验证。

ON 使用原生自动 host fraction0.65、write_through、kernel 和 page_first，不设置 size/ratio，不增加自定义 guard 或引擎补丁。dram-utilization: 1.0 经现有历史上限和四卡份额计算得到 1499 GB 名义元数据，不是实际预留或 pool 分配；原生 allocator 可能看到整台独占节点的可用 host 内存，必须记录实际联合 pool 和 cgroup 限制。

保留官方固定镜像、模型 revision、NVFP4 target、原始 draft 加载路径、禁用转换标志、完整 1,048,576-token 上下文、393-entry AgentX 数据集和 3600 秒时长，性能使用共享实测 AL 3.603536。新增 TP4 的 OFF c16 和 ON c32 独立 canonical eval，均为完整 1319 题、关闭 synthetic acceptance；使用不同 eval 并发避免现有标识冲突。原 TP8 配置及 ON c32 eval 均不修改。

本 PR 保持实验性 draft，无 GPU sweep 标签。全量运行前必须检查实际分配并通过关闭 synthetic acceptance 的真实 host eviction/restore 验证；启动成功并不能证明 restore 正确。既有空响应错误仍未解决，当前不宣称 B300 TP4 已通过运行或已改善前沿。

AI model disclosure

GPT-6 Astra coordinated and implemented this TP4 child. Delegated GPT-6 Astra agents supplied exact-build GB200/GB300 TP4 runtime evidence, HiCache source and budget audits, evaluation-plan checks, and active-versus-reserved allocation accounting.

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=41353165651

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=41353165651

[Oseltamivir]

/use 41353165651

[github-actions]

@Oseltamivir staging run 41353165651. Existing staged runs will be preserved; a completion link will be posted here.

[Oseltamivir]

Superseded by #4710 (merged as 23a74c6), which consolidates GLM-5.3-Flash B300 FP8 + NVFP4 base and HiCache points into one PR (no TP4 on the 8-GPU node). / 已由 #4710（合并为 23a74c6）取代，该 PR 将 GLM-5.3-Flash B300 FP8 与 NVFP4 的基础点和 HiCache 点合并为一个 PR（8 卡节点不跑 TP4）。