diff --git a/.github/workflows/h3-media.yml b/.github/workflows/h3-media.yml index 5c02a85d4..5a8adfdae 100644 --- a/.github/workflows/h3-media.yml +++ b/.github/workflows/h3-media.yml @@ -1,6 +1,8 @@ name: 'Publish H3 media' on: + repository_dispatch: + types: [publish-video-results] schedule: - cron: '*/15 * * * *' workflow_dispatch: @@ -31,5 +33,5 @@ jobs: bun-version-file: package.json - name: Persist completed H3 artifacts env: - H3_RUN_ID: ${{ inputs.run_id }} + H3_RUN_ID: ${{ inputs.run_id || github.event.client_payload.source_run_id }} run: bun packages/app/scripts/warm-video-results.ts diff --git a/.prettierignore b/.prettierignore index 7567a05df..5fbab2051 100644 --- a/.prettierignore +++ b/.prettierignore @@ -11,3 +11,7 @@ packages/app/cypress/fixtures/api/collectivex-runs.json # The power-model parity fixture is upstream Python output, regenerated wholesale by # packages/app/scripts/generate-power-model-parity.py; pretty-printed it is ~5x larger. packages/app/src/lib/power/power-model.parity.json +# The native H3 bundle fixture is a checksum-sealed copy of the InferenceX producer's +# output: SHA256SUMS and the manifest bind every byte, so reformatting breaks the seal. +packages/app/src/components/video-benchmark/fixtures/native-serving-bundle/ +packages/app/src/components/video-benchmark/fixtures/native-startup-failure/ diff --git a/docs/dashboard-readonly-views.md b/docs/dashboard-readonly-views.md index b8a2a82a7..77879cec2 100644 --- a/docs/dashboard-readonly-views.md +++ b/docs/dashboard-readonly-views.md @@ -75,7 +75,7 @@ combinations, not the full Cartesian product of all possible filter values. | `reliability` | `asOf`, `format`, `gpus`, `range` | | `submissions` | `direction`, `limit`, `lines`, `mode`, `offset`, `onChangeOnly`, `search`, `sort` | | `ubenchx` | `gpu`, `test` | -| `video` | `artifact`, `cell`, `compare`, `costs`, `gpuBasis`, `page`, `phase`, `run`, `selected`, `slot`, `source`, `view`, `workload`, `xAxis`, `yAxis` | +| `video` | `artifact`, `cell`, `compare`, `costs`, `gpuBasis`, `page`, `phase`, `run`, `selected`, `slot`, `source`, `v_x`, `v_y`, `v_tier`, `v_view`, `v_optimal`, `v_api`, `v_hidden`, `v_base`, `v_cand`, `v_case`, `view`, `workload`, `xAxis`, `yAxis` | Cache reuse returns `data.recipes` and the resolved `params.recipe`. Pass a returned key as `recipe` (the UI uses `c_recipe`) to select the same TP/EP/DP-attention, worker, GPU, speculation, offload and fingerprint combination. Omitted or stale keys use the shared dashboard default. Runs use that recipe if available, otherwise their own best-covered recipe; inspect each bar's source row for its identity. Overlay-only configurations expose their own recipe choices. Layout orientation and label placement are presentation-only controls. @@ -120,9 +120,74 @@ alignment changes no API parameters, response schemas, or stored model identitie - GPU metrics: shared line/correlation transforms and full-record per-GPU statistics computed from the stored samples, or from the samples of live artifacts. File/host identity and missing-versus-zero semantics persist. -- Video: checksum-verified stored bundles, serving/fidelity selectors and tradeoffs. +- Video: published history pages → videoPoints → dashboardCells → shared metrics, + deployment Pareto plot/table, lead-cell KPIs, Compare and measured evidence. + Legacy checksum-verified bundle serving/fidelity and tradeoff reads remain available. - Overview/rankings/compare: existing discovery-page assembly and scenario helpers. +## VideoGenX dashboard contract + +`GET /api/v1/views/video` defaults to the current `/video` dashboard. The shared +`VIDEO_HISTORY_MAX_PAGES` bounds both UI and API to the first five published +history pages; responses report `coverage` with `pagesRead`, `maxPages`, +`nextPage` and `truncated`. This is a publication-ordered window, not an execution +date range or a promise to enumerate the entire catalog. Failures remain failures; +missing fields remain null. Source run/artifact IDs, seals and distinct execution +and publication times remain in `provenance`; raw documents, private asset URLs +and credentials are excluded from the dashboard projection. + +| UI control or surface | Public projection / scope | +| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| X/Y, cost tier, optimal | `v_x`, `v_y`, `v_quality`, `v_qmin`, `v_tier`, `v_optimal`; same metric registry, participating-GPU cost basis and per-hardware frontier helpers. Queueing cells never enter `plot`/`rows`. | +| Quality view/filter | `v_y=quality`, `v_quality` (seven original human dimensions) and optional `v_qmin` (0–4). Require explicit ordinal_0_to_4 scale and calibrated pass for all seven critical dimensions, complete assessed coverage matching the point, protocol/rubric/evaluator hashes and frozen calibration provenance before Pareto selection. The reader filter cannot relax the frozen threshold. Chart/table/CSV/API agree; raw judged-but-unqualified scores remain distinct from derived eligible values. | +| Hardware visibility | `v_hidden`, sorted/deduplicated known roster keys; affects plot/table only, matching UI. KPI, Compare and Evidence retain all selected-workload cells. | +| API reference price | `v_api`, positive USD/video-second rounded to four decimals; only `apiPricePerVideo`, never TCO or revenue. Invalid values use the UI reference. | +| Chart/table | `v_view`; echoed as `params.displayView`, no numerical difference. Color, zoom and playback remain presentation state. | +| Workload/newest cell | `videoPoints` → `dashboardCells`; same canonical identity, sample floor and publication precedence. Return `workload`, `otherWorkloads`, cells and source provenance. | +| KPI cards | `kpis`: roster entries, shared lead cell and all metric values; missing hardware keeps null point/metrics. | +| Compare selectors/deltas | `v_base`, `v_cand`, `v_case`; shared selection defaults and `compareMetrics`. Default mode returns deltas without loading media. | +| Case media metadata | `view=compare` adds at most two published-only `format=published` reads (deduplicated by run/artifact), `compareSide`/`pairCases`, bounded `resolvedCaseIndex`, matched/unmatched counts and selected records. Only bundle-relative media paths are exposed. Missing media, missing baseline/candidate and read errors have separate statuses; `ready` means readable artifacts and can still have zero pairs. | +| Serving outcomes | `serving` retains every loaded planned cell before chart/workload/hardware/quality filters, including zero-attempt failures. Attempted failures and not-started slots remain separate; timeouts are a failure subset. Unknown populations remain null, and `qualitySloGoodput` is null without per-request quality/deadline evidence. | +| Evidence | Shared power/plateau/scaling/facts functions. Descriptive measurements; no quality qualification or causal bottleneck inference. | +| Legacy run browser | Explicit `view=discovery&page=…`; supplying `run`/`artifact` retains results and optional tradeoff selectors. Mixing legacy selectors and `v_*` returns 400. | + +Dashboard state values use the UI parser and its fallback defaults; unknown or +repeated parameter names return 400. Legacy source status handling and 204 behavior +remain unchanged. Chart history read failures preserve the sanitized upstream status; +optional Compare media failure returns `comparison.cases.status=unavailable` without +erasing independently available chart data. The route and generated OpenAPI remain +no-store. + +A measured frontier is grouped by explicit workload/generation identity and hardware; +quality mode additionally requires the same contract, dimension, evaluator and +calibration cohort. Hardware-health failures and unknown request capacity are +excluded from ordinary performance rankings. A single point remains a point; +lines join measured eligible points and do not extrapolate. `quality: null` +means unjudged, and quality modes return an honest empty selection until calibrated +evidence exists. Integrity checks and numerical fidelity are separate evidence. + +Regression coverage exercises numerical API/UI parity across default and changed +axes/tier/optimal/price/visibility, the shared five-page bound, legacy reads, +case-index clamping, absent/failed media and private-URL exclusion. This does not +claim live production API/UI equivalence or full-history completeness. + +### 中文说明 + +`v_y=quality` 展示单项人工质量评分与延迟,`v_quality` 选择七个独立维度之一, +`v_qmin` 在 Pareto 计算前按 0–4 级阈值筛选。必须具备完整样本覆盖、已校准的通过 +判定及协议、rubric、evaluator 和校准来源记录;读者阈值不能放宽冻结的规则。 +未评价或未校准数据保留为空,图表、表格、CSV 和 API 选择一致。原始已评但未获 +资格的分数可作为证据读取,不能进入质量前沿。硬件健康失败及未知 batch 容量不会 +进入普通性能排名;不同 workload、生成配置或质量 evaluator 不合并连线。 + +视频公开接口默认复现当前仪表板:最多读取前五页已发布历史,复用相同的 workload、 +样本门槛、部署和数值计算,明确返回覆盖范围与截断状态。`v_hidden` 只隐藏图表和表格 +中的硬件,KPI、Compare 和 Evidence 保持原范围;`v_api` 只是视频秒参考价,不是 +TCO 或收益。默认不加载媒体;`view=compare` 才读取最多两份已发布媒体产物并返回 +匹配用例、实际索引和未配对数量,不暴露 asset URL 或原始数据包。`view=discovery` +及原有 run/artifact 查询仍可使用;不能与 `v_*` 混用。未发布、无可比较双方和读取失败 +分别标记,不能解释为成功的空测量。该接口不承诺完整历史、质量验收或瓶颈因果结论。 + ## Other public surfaces Model, chip and pair discovery pages reuse overview/rankings/compare calculations. @@ -455,3 +520,9 @@ combinations from pending qualification. Footnotes contain both `en` and `zh`. CollectiveX 视图另返回 `knownSupport` 和 `knownSupportFootnotes`,与网站共用人工维护 的数据源,不受所选运行影响。状态值为 `works`、`broken` 和 `na`;请结合注释区分 不可用与待验证的配置。注释同时包含 `en` 和 `zh`。 + +### Video engine identity + +Published history exposes `producer` when the manifest, matrix and execution records agree on the selected `plan_sha256`, model revision and `engine: {name, revision}`. Legacy sources retain null identity. The dashboard and API expose `releaseCoverage` for H3/Wan × SGLang/vLLM-Omni across the loaded published pages, including missing, failed, history-only and sample-floor-met states. This bounded coverage is not a full-catalog or quality qualification claim. Native four-clip runs remain History-only; the chart, KPI and Compare sample floor stays ten. + +Compare keeps distinct engine revisions on the same GPU. `v_base` and `v_cand` accept `hardware:engine:revision`; hardware-only links remain valid. Technical deltas require the same sealed model revision, selected plan and generation settings. Unknown quality does not suppress technical observations, but does not authorize quality-normalized claims. CSV rows include `model`, `model_revision`, `engine_revision` and `plan_sha256`. Compare reads `format=published` only. diff --git a/docs/index.md b/docs/index.md index 80256044e..c902033a9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,6 +22,7 @@ Design rationale and non-obvious conventions. See [CLAUDE.md](../CLAUDE.md) for - [Architecture](./architecture.md) — Why client-first, route navigation, URL state, provider nesting, server-side caching (unstable_cache + blob), in-memory client cache, color system, analytics enforcement - [D3 Charts](./d3-charts.md) — Why 4 effects, in-place mutation, refs for zoom, rAF throttling, HTML tooltips, Pareto directions, gradient labels +- [Video History](./video-history.md) — Read-only published H3 catalog, metric boundaries, sharing, and local retained-data replay - [Data Pipeline](./data-pipeline.md) — DB schema reasoning, ETL design, transform pipeline, spline method choice, normalizer resolution order (model/GPU/framework) - [Pitfalls](./pitfalls.md) — Failure modes: token type consistency, schema evolution, empty objects, zoom loss, stale closures, disaggregated metrics, negative splines, date stamping, ref stability, cost inheritance - [GPU Specs](./gpu-specs.md) — Unit conventions, topology invariants, SVG layout rationale, hardware gotchas @@ -34,6 +35,7 @@ Design rationale and non-obvious conventions. See [CLAUDE.md](../CLAUDE.md) for - [State Ownership](./state-ownership.md) — Which context owns which state, availability filtering cascade, comparison date mechanics, URL param sync - [Blog](./blog.md) — MDX content system, SEO features (OG images, RSS, llms.txt, JSON-LD), TOC sidebar, reading progress, heading links, analytics events - [CollectiveX](./collectivex.md) — lazy ingest-on-read pipeline (separate Neon DB as a durable GitHub-artifact cache), tombstoned deletes, raw-docs storage + shared reader +- [VideoGenX dashboard](./videogenx-dashboard.md) — `/video` cross-hardware chart: published-history projection → points → metrics, TCO tiers per participating GPU vs. the API list price, deployment cells (queued cells never plotted), arena Compare, performance evidence, unavailable-hardware roster, null rules - [Chinese Pages (/zh)](./i18n.md) — Why hand-authored /zh pages instead of an i18n framework, hreflang pairing, blog translation pairing, html lang workaround, CJK reading time/slugs - [Chinese Copy Editorial Guide](./chinese-copy.md) — Audience, surface-specific register, context-aware terminology, rewrite workflow, manual-review process, and PR checklist diff --git a/docs/video-history.md b/docs/video-history.md new file mode 100644 index 000000000..b70686966 --- /dev/null +++ b/docs/video-history.md @@ -0,0 +1,112 @@ +# Video performance history + +`/video` and `/zh/video` lead with the cross-hardware dashboard (see [VideoGenX dashboard](./videogenx-dashboard.md)); the **Performance history** list described here sits in a collapsible section below it, which opens automatically when the URL carries a `history-*` filter (`history-hardware`, `history-concurrency`, `history-query`). Each entry links to its original GitHub Actions run. The earlier per-run viewer, trade-off chart and CI-run browser were removed with the dashboard rebuild; clips are compared in the dashboard's Compare panel. + +## Read path + +`GET /api/video-runs?format=history&page=1` returns a versioned UI projection of existing published video indexes. It lists `h3-video-media/v1/runs/` metadata, validates pathname identities, sorts by index publication time (artifact ID breaks ties), then reads at most ten indexes for one page. It neither downloads media nor calls the publication path. The repository visibility check still applies. This UI transport is not a stable public benchmark API. + +The projection groups all cells under the original source inside one artifact entry. It reuses `storedBundle`, `servingCells`, `storedFidelityBundle`, and the existing tradeoff calculations. Invalid or unavailable indexes remain explicit error entries. Original execution time comes from `ci.started_at`; index publication time is separate and must never be substituted for execution time. Original run IDs and export run IDs remain distinct. + +Two launchers produce the serving matrix, and `serving.ts` keys the contract on the manifest's `evidence_kind`. The legacy controlled GPU supervisor (`controlled_h3_gpu`) wrote `controlled_serving_smoke` receipts, device inventories, power-limit snapshots and fully timed request records. The native srt-slurm launcher (`srt_slurm_h3_service`, InferenceX #2916) writes `srt_slurm_h3_service_receipt` receipts that seal a `h3_native_power` document (`power.valid`, the receipt's `power.sha256` and the measurement phase must agree), per-cell InferenceX sidecars (`aggregate.json`, `benchmark_concurrency_.json`; a `power_valid` of 0 or an `energy_normalization` other than `clips` withholds the number), and a matrix-level `power_verified` verdict shown on the source line. Native bundles name no GPU model: the projection reads `ci.site.gpu_model` when present, else the registry SKU in `ci.ci.runner_name`, else leaves the hardware empty. When an srt-client run record carries no per-request latency, the harness summary (`latency_median_seconds`, `valid_clips_per_second`, cross-checked against the matrix cell) stands in for P50 and throughput; P90 still needs the per-request population. Power that is not `valid` renders as not measured, never as a number. + +Allocated GPU counts require retained Slurm `AllocTRES` or the validated AMD allocation binding; a native power document's `expected_gpu_count` describes telemetry coverage and cannot establish the reservation. Without allocation evidence, the count and allocation-normalized metrics remain null. + +Each observation also carries the cell's participating and allocated GPU counts, measurement wall seconds, clip duration and frame count, summed mean board watts, summed recorded enforced limits (null when no limit snapshot exists), the configured `tp_size` / `ulysses_degree` / `attention_backend` and the endpoint's replica count (`execution.deployment.replica_count`, null for bundles that predate it). The nullable `workloadKey` carries the full canonical workload identity when complete; prompts and seeds cannot be replaced by the shortened display label. These are additive, nullable fields under the same `schemaVersion: 1`; the hardware dashboard reads at most five pages and derives every displayed metric from them, so the browser never parses an artifact bundle for the chart. + +## Publication and discovery + +The `publish-video-results` repository dispatch starts publication for its `source_run_id`; scheduled and manual catch-up use the same job. The publication job follows every catalog `nextPage`, including empty filtered pages, and deduplicates run/artifact IDs. Each per-run lookup reads all GitHub artifact pages. Standard `run-sweep.yml` runs join legacy H3 workflows in discovery; a sweep with no video artifact remains empty. An explicit `H3_RUN_ID` limits catch-up to one run. Failed artifacts do not prevent later artifacts from being attempted, but the job still fails; timeout or failure is not proof that the catalog was fully processed. + +Native matrix artifacts use `h3-results---` while legacy names remain readable. The sealed `artifact_name`, run/attempt, and `ci.head_sha` bind a new artifact to GitHub metadata. The actual benchmark checkout remains `git_commit`, which may differ from the Actions head when CI tests a merge commit. Publishing still writes immutable objects before the index; history and Compare only read published indexes. Low-sample and failed sources remain history entries, without changing chart admission. + +The additive point contract also retains deployment/generation settings, hardware-health status, nullable per-dimension `quality`, original source ID/manifest SHA/source revision/artifact digest and independent fidelity/calibration/release labels. Old records with absent fields remain null. Quality scores require point-linked evaluator, rubric and calibration provenance; a history-level calibration label does not qualify its performance observations. + +## Producer model identity + +Serving imports accept `video-serving--` alongside legacy H3 artifact names, +using the same immutable `h3-video-media/v1` namespace and checksum checks. The generic +serving matrix/evidence names are `video_serving_smoke_matrix`, `controlled_video_gpu` and +`live_video`; these generic supervised evidence tags must match the Wan model. Native selected-plan bundles keep H3 kind constants for both models. H3 retains its legacy tags; both accept `operator_endpoint` request records under a verified model-specific supervisor. Paired H3 result/fidelity contracts are unchanged. Wan request duration derives from `frame_count / fps`; H3 retains its explicit requested duration. +`manifest.workload_plan.model_id` preserves a failed producer's identity without `result.json`. +Wan's canonical ID is `Wan-AI/Wan2.2-T2V-A14B-Diffusers`; the old short ID is a selection alias. +A contradictory result identity is rejected; trusted history retains the manifest identity and +an explicit error, never an H3 fallback. Explicit planned counts require a sealed initial +`serving-smoke.json`; absent counts stay null. Video-only media may omit audio, while the +existing GPU verification, workload, media hash and sample requirements still govern charts. + +## Source-bound observation metadata + +A producer may add `dashboard-observations.json` with exactly +`{schemaVersion: 1, sourceRunId, sourceSha, cells}`. `sourceRunId` and `sourceSha` +must match the source manifest's `run_id` and `git_commit`. Its SHA256 must match +both the bundle checksum inventory and `manifest.evidence["dashboard-observations.json"]`; +`readVerifiedFiles` verifies the raw bytes during ingestion. Retained `StoredArtifact` +objects are the trusted transport of those already-verified documents and seals. + +`cells` is a nonempty map keyed by existing serving cell IDs. Each cell records +`runSha256` and `specSha256`, matching that cell's sealed run/spec files, plus +optional nullable `deployment`, `hardwareHealth` and `quality`. The strict parser +rejects unknown fields, wrong types, invalid hashes and source/cell mismatches. +An invalid sidecar rejects that source's observation projection; an absent +sidecar preserves null additive fields. The nullable field shapes are documented +in the public video response schema; they do not alter metric units or turn +uncalibrated media similarity into quality qualification. + +A local curator can enrich a development replay through the same field parser. +This remains an explicitly unsealed projection: it does not modify the sealed +source artifacts or acquire producer attestation by passing type validation. + +## Evidence boundaries + +- Only descriptive observations are shown: this version has no selected matched baseline and computes no before/after delta. +- Valid/completed/scheduled/failed counts stay separate. Missing or invalid metrics stay null. P90 retains the existing per-cell ten-sample display floor. +- Latency and throughput use the submission-to-downloaded-media window. Energy uses the recorded generation window and participating GPU boards. Board energy is not facility energy. +- Fidelity, calibration and release qualification are displayed independently. Stored/published does not mean qualified. +- Filters apply to loaded pages. The user can load older pages without losing already loaded entries. Global execution-time sorting and an editorial change/PR catalog require a later materialized catalog; the current implementation scans index metadata only, never all full indexes. + +## Local replay + +`E2E_FIXTURES=1` serves `cypress/fixtures/api/video-history.json` through the existing fixture loader and checksum manifest. The fixture is a metadata-only projection of retained H100/H200/B200 observations and two fidelity artifacts; it contains no infrastructure paths or video bytes. Unknown publication timestamps remain null. The response header `X-VideoGenX-Replay: 1` causes a visible replay notice. These are historical retained observations, not newly measured results or a new publication. + +For a developer's retained-data preview, `VIDEOGENX_REPLAY_DIR` supplies a local `history.json` plus `..json` metadata files through the same handlers only with `NODE_ENV=development`. This separate replay directory keeps private receipts and local paths outside the committed fixture and public catalog. It serves retained metadata, not new measurements or a production publication; unavailable media stays unavailable. The matching public view projection uses the same history and eligibility helpers. + +## 中文说明 + +`publish-video-results` repository dispatch 为指定 `source_run_id` 启动发布;定时任务和手动补发布复用同一任务。发布任务跟随所有 `nextPage`,包括筛选后为空的页面,并对运行和产物 ID 去重。单次运行会读取 GitHub 产物列表的全部分页;标准 `run-sweep.yml` 也进入发现列表,没有视频产物的 sweep 保持为空。`H3_RUN_ID` 可将补发布限定到一个运行。某个产物失败后仍尝试后续产物,但任务最终报告失败;超时或失败不代表完整处理了目录。 + +原生矩阵采用 `h3-results---`,旧名称仍可读取。封存的 `artifact_name`、run/attempt 和 `ci.head_sha` 对应 GitHub 元数据;实际 benchmark checkout 仍单独记录为 `git_commit`,测试 merge commit 时它可以不同于 Actions head。发布继续先写不可变对象、最后写索引;历史和 Compare 只读已发布索引。失败或低样本来源保留在历史中,不降低图表门槛。 + +`/video` 与 `/zh/video` 首屏为跨硬件仪表板(见 [VideoGenX 仪表板](./videogenx-dashboard.md));本文描述的“性能历史”列表位于其下方可展开的区块,URL 带有 `history-*` 筛选参数(`history-hardware`、`history-concurrency`、`history-query`)时自动展开。每条记录链接到原始 GitHub Actions 运行。原有的按运行查看结果、trade-off 图表与 CI 运行浏览已随仪表板重建移除;视频对比由仪表板的 Compare 面板提供。 + +历史读取只列举既有发布索引的元数据,按索引发布时间排序,每页最多读取十份完整索引;不下载或发布媒体。原始执行时间取自 `ci.started_at`,与发布时间、重新导出的时间分开。索引无效时保留错误记录,缺失指标保留 null。已分配 GPU 数须有保留的 Slurm `AllocTRES` 或通过验证的 AMD allocation binding;原生功率文档的 `expected_gpu_count` 仅描述遥测覆盖范围,不能证明预留数量。缺少分配证据时,该数量及按分配数量归一化的指标保持 null。每个 cell 另附参与/已分配 GPU 数、测量窗口秒数、视频时长与帧数、板卡平均功率之和、记录的生效功率上限之和(无快照时为 null)、配置的 `tp_size` / `ulysses_degree` / `attention_backend`,以及端点的副本数(`execution.deployment.replica_count`,早于该记录的产物包为 null);`workloadKey` 在信息完整时保留完整工作负载身份,prompts 与 seeds 不能由缩写显示标签替代。这些都是同一 `schemaVersion: 1` 下可为空的附加字段,硬件仪表板最多读取五页,并据此推导全部指标,浏览器不再解析产物包。 + +serving 矩阵有两种来源,`serving.ts` 按 manifest 的 `evidence_kind` 区分契约:旧的受控 GPU supervisor(`controlled_h3_gpu`)写入 `controlled_serving_smoke` 回执、设备清单、功率上限快照和带完整计时的请求记录;原生 srt-slurm launcher(`srt_slurm_h3_service`,InferenceX #2916)写入 `srt_slurm_h3_service_receipt` 回执,并对 `h3_native_power` 功率文档做封印(`power.valid`、回执中的 `power.sha256` 与测量阶段必须一致),同时附带每个 cell 的 InferenceX sidecar(`aggregate.json`、`benchmark_concurrency_.json`;`power_valid` 为 0 或 `energy_normalization` 不是 `clips` 时不显示对应数值),矩阵级的 `power_verified` 结论显示在来源行。原生产物包不记录 GPU 型号:投影优先读取 `ci.site.gpu_model`,其次从 `ci.ci.runner_name` 中识别注册表 SKU,都没有时硬件留空。srt-client 的运行记录没有逐请求延迟时,以 harness 汇总(`latency_median_seconds`、`valid_clips_per_second`,并与矩阵 cell 交叉核对)代替 P50 与吞吐量;P90 仍需要逐请求样本。功率未通过 `valid` 校验时一律显示为未测得,不显示数字。 + +当前展示观测值,未选择匹配基线,也不计算 before/after。延迟、吞吐量使用提交到下载完成的窗口;板卡能耗使用记录的生成窗口。保真度、阈值校准、发布验收单独展示。筛选仅作用于已加载的页面,可继续加载较早结果。 + +`E2E_FIXTURES=1` 使用保留结果的脱敏元数据回放,遵循现有 fixture 校验和清单,并显示回放提示。未知发布时间保持为空;这不是新测量或新发布。 + +附加字段还保留完整部署/生成配置、硬件健康、逐维度 `quality`、source ID、manifest SHA、source revision 与 artifact digest;旧记录缺少的字段仍为 null。历史层的校准标签不能代替逐点质量证据。质量未评价或未校准时不会进入质量前沿。 + +开发环境可通过 `VIDEOGENX_REPLAY_DIR` 读取本地 `history.json` 和 `..json` 元数据;生产环境禁用该入口。保留数据目录与已提交的回归 fixture、公有结果目录相互独立,缺失媒体不会被标为已发布。 + +可选 `dashboard-observations.json` 使用 `schemaVersion: 1`,其 `sourceRunId`、 +`sourceSha` 必须匹配原始 manifest,文件 SHA256 同时匹配 checksum 清单及 +`manifest.evidence`。`cells` 按已有 serving cell ID 保存对应 run/spec 的 SHA256 +和可为空的 deployment、hardwareHealth、quality。未知字段、类型错误或身份不符 +会拒绝该 source 的观测投影;无 sidecar 时附加字段保持 null。开发回放的人工补充 +也经过字段解析,但仍是明确未封存的投影,不会改写原产物或获得 producer attestation。 + +通用 serving 产物名称为 `video-serving--`,沿用 `h3-video-media/v1` +及原有校验流程。`video_serving_smoke_matrix`、`controlled_video_gpu`、`live_video` +与旧 H3 类型兼容,但证据类型须与模型一致。两种模型均可在对应 supervisor 验证后使用 `operator_endpoint` 请求记录。配对结果和保真度契约保持不变。Wan 按 `frame_count / fps` 计算请求时长,H3 保留显式请求时长。即使没有 `result.json`, +`manifest.workload_plan.model_id` 也能保留失败来源的模型身份。Wan 的规范 ID 为 +`Wan-AI/Wan2.2-T2V-A14B-Diffusers`,旧短 ID 仍可用于模型选择。结果中的模型与 +manifest 冲突时会报错,不会归入 H3。计划计数须来自已封存的初始 `serving-smoke.json`, +未知值保持 null。纯视频产物可不含音频;进入图表仍须满足原有 GPU 执行、工作负载、 +媒体哈希和样本数要求。 + +### Native model and engine provenance + +Native selected-plan bundles keep the legacy H3 bundle, receipt, matrix and power kind constants for both H3 and Wan. The sealed workload model identifies the model; a filename prefix does not. New `producer` projections validate the manifest and matrix engine name/revision and selected plan SHA against each retained run and job. The selected YAML plan hash is separate from the execution spec hash. Failed sources can retain their sealed requested model/engine identity without a successful measurement. diff --git a/docs/videogenx-dashboard.md b/docs/videogenx-dashboard.md new file mode 100644 index 000000000..adcc50e12 --- /dev/null +++ b/docs/videogenx-dashboard.md @@ -0,0 +1,114 @@ +# VideoGenX dashboard (/video) + +Why `/video` leads with a cross-hardware chart instead of a CI-run browser, and the rules its metrics obey. Component code lives in `packages/app/src/components/video-benchmark/` (`VideoDashboard.tsx` is the root). + +## Data path + +Published blob indexes → `GET /api/video-runs?format=history` (the per-cell projection described in [Video performance history](./video-history.md), with additive nullable fields for GPU counts, wall time, clip shape, board power, enforced limit and server layout) → `points.ts` (`videoPoints`, `latestVideoCells`) → `metrics.ts` (`metricValue`). The browser reads at most five history pages and never parses an artifact bundle for the chart. The previous CI-run viewer has been removed: `history-*` links open Performance history; Compare loads published media on demand. The public `/api/v1/views/video` dashboard projection reuses these selectors and formulas; explicit legacy views remain available through the API. + +## Axes and defaults + +`video-url-state.ts` owns the `v_*` URL params (`v_x`, `v_y`, `v_quality`, `v_qmin`, `v_tier`, `v_view`, `v_optimal`, `v_api`, `v_hidden`); defaults are omitted from the URL and every other param is left alone. + +- **X = P90 time to video (s)**, lower is better; `p50Latency` is the alternative. +- **Y = videos per $1 TCO** at the _Owning at Large Hyperscaler Volume_ tier; TCO cost per video, videos per GPU-hour and GPU-board energy per video (kJ) are the alternatives. Cost per GPU-hour comes from `HW_REGISTRY` (`costh`/`costr`), the same SemiAnalysis TCO source `/inference` uses; the badge row and source link sit above the chart. +- **Per participating GPU**: every per-GPU metric divides by the boards that generated the clip (4 per video with TP2 × Ulysses 2), never by the boards a job reserved. The retained H100 and B200 jobs reserved eight and used four, so a note under the chart header names them ("B200 (4 of 8), H100 (4 of 8) … the idle boards are not counted") and their KPI card headers read "4 of 8 GPUs". Idle boards are stated, not billed; there is no GPU-basis toggle. +- **Quality**: `v_y=quality` uses one human rubric dimension selected by `v_quality`; `v_qmin` optionally filters that dimension before frontier computation on any Y axis. Defaults stay P90 versus videos per $1 TCO; the retained deployment preview explicitly selects `v_y=videosPerGpuHour`. The seven original dimensions are prompt adherence, visual fidelity, temporal consistency, motion plausibility, audio quality, audio content and audiovisual synchronization. All preserve higher-is-better 0–4 ratings, never an overall mean. Quality admission requires complete calibrated evidence for every critical dimension; the selected dimension supplies only the Y coordinate or extra reader threshold. See the qualification boundary below. +- **Both axes start at zero** (`chart-domain.ts`), so per-dollar and per-video readings compare by length, not by offset. +- **Chrome mirrors `/inference`**: a `Card` with `DashboardSectionHeader` (title, description, Share) sits above the two config panels; the cost tier is picked inline in the chart caption (`video-cost-tier`, a `SearchableSelect` with an `export-only` text twin) like the inference caption's Cost Tier selector; the Compare and performance evidence sections fold behind a chevron (`CollapsibleSection`), and Compare opens on load for `v_base`/`v_cand`/`v_case` deep links. The video route declares `shareParamScopes: ['v_']`, and `collectTabParams` reads that scope live from the address bar, so Share links carry the current `v_` params. Hidden hardware is canonicalized in `v_hidden`; it filters the chart, table and CSV while KPI cards, Compare and evidence retain the selected workload. + +## Deployments, not concurrency, make the curve + +`/inference` sweeps batch size to trace each hardware's latency-vs-throughput frontier. The retained fixture uses a batch-one, single-replica H3 server: its C1/C2/C4 cells show throughput within ±1.5 % while latency grows with C. This describes that recorded configuration; it is not a claim about every diffusion server or future runtime. Joining those cells would draw a flat "curve" that measures the queue, not the hardware, so the chart never does. + +A measured deployment is identified by participating GPUs, GPUs per replica, replica count, TP/Ulysses/Ring/CFG layout, attention, offload, encoder parallelism, actual/configured batch settings, scheduling, engine, precision, acceleration and generation identity when recorded. Missing settings stay null. Distinct identities remain distinct cells. The legacy batch-one single-endpoint fallback applies only to old records; modern batching records require known capacity. Unknown dynamic-batch capacity and recorded hardware-health failures are excluded from normal performance rankings. + +`plot.ts` filters eligible deployment observations before computing the per-hardware frontier. Comparisons require an explicit canonical workload and compatible generation settings; quality mode additionally groups by contract, rubric, dimension, evaluator and calibration identity. A single measurement is a point. Lines connect measured frontier points only, without interpolation or extrapolation. Optimal Only preserves tied nondominated configurations. Chart, table, CSV and the public API share the selection and metric functions. + +The committed regression fixture retains the historical single-deployment-per-hardware scatter. A development-only retained-data replay can additionally expose verified H200/B200 2/4/8-GPU deployments without replacing that fixture, publishing data or rerunning generation. + +## Quality evidence and empty state + +Goal 1's H3 Quality Contract v0 defines a blinded seven-dimension 0–4 rubric but has no completed judgments or accepted thresholds. Existing integrity checks, PSNR and audio spectral similarity remain separate technical evidence. The app uses the original rubric IDs with an explicit `scale: ordinal_0_to_4` declaration and registry version `0.2.0-draft`. Legacy 1–5 or missing scale declarations are not converted or admitted. All critical dimensions retain their own evaluator, calibration and frozen rule in cohort identity. A/A diagnoses numerical repeatability; it does not calibrate acceptable perceptual quality. + +`quality.ts` requires a `pass` for each of the seven critical dimensions, finite in-range value, protocol/rubric/evaluator identities and hashes, complete assessed-clip coverage matching the point, and a calibrated frozen threshold with cohort, time and provenance. It checks every critical dimension before the reader's optional threshold on the selected dimension. The reader cannot weaken the frozen rule. Raw `judged_unqualified`, failed or partial assessments can remain in point metadata but have a null eligible value. Unknown or uncalibrated data produces a useful empty quality view; no metric is imputed from media validity. + +Point details and API records retain deployment/generation settings, sample/failure counts, source/run/artifact identities and available evidence links. Modelled TCO assumptions remain separate from measured GPU-board energy. Generation-setting changes remain explicit comparison cohorts; a speed observation alone cannot establish equivalent quality. + +## Sections below the chart + +- **API price reference** (`api-reference.ts`, `VideoApiReference.tsx`): a dated, editable USD per video-second list price (`v_api`, default `H3_API_REFERENCE`) feeds exactly one derived number, the API list price per video (`apiPricePerVideo` = price × clip seconds). It is shown beside the TCO cost per video on the KPI cards and as a table and CSV column; the chart never plots it, and no revenue, profit or price multiple is derived from it. The default records the MiniMax platform pay-as-you-go page captured on 2026-09-24 (the source has no effective date); it is a dated list-price input, not a current price guarantee; a null or non-positive price nulls the list price, and `parseApiPrice` falls back to the reference rather than pricing at 0. +- **Compare** (`compare.ts`, `compare-url-state.ts`, `VideoCompare.tsx`): arena-style baseline vs. candidate, video first — the same prompt + seed clips from both CI runs play side by side with shared play/pause/restart and previous/next case arrows (no blind mode, no case dropdown); the deltas below are fixed to time to video (P50), TCO cost per video and energy per video (`COMPARE_METRICS`, registry polarity), with the cost tier stated in a footnote. One selectable lead deployment per hardware and sealed engine revision, defaulting to the slowest hardware as baseline and the fastest as candidate; `v_base`, `v_cand`, `v_case` in the URL. The two stored artifacts (several MB each) load once the panel scrolls into view; an HTTP 204 side reads as "clips not published", never as an error. +- **Performance evidence** (`evidence.ts`, `VideoEvidence.tsx`): recorded board-power ratios, concurrency throughput/latency ratios and observed speed-ups beside memory-bandwidth/FLOPS specifications. Cross-hardware rows require a known canonical workload, a complete matching deployment layout and C1 (`sharedLayoutCells`); unmatched hardware is omitted, never replaced with its lead cell. Concurrency ratios require a matching workload, layout and replica count. The copy reports numbers without inferring power saturation, a throughput plateau or a compute/memory bottleneck. These mechanisms need separate experiments and telemetry. + +## Rules + +- **Unavailable hardware** stays visible. `VIDEO_HARDWARE_ROSTER` in `hardware.ts` lists the campaign targets; MI355X has no valid run and renders as a muted legend row, a "Not measured" KPI card with the failed-run link, and no table row. A missing point is never a zero. +- **Nulls**: every formula returns `null` when an input is missing or invalid — power phase invalid, fewer than ten samples for P90, unknown hardware key, missing GPU count. `formatMetric` prints `—`. +- **Newest observation wins**: `latestVideoCells` keeps the first (newest publication) observation per hardware × deployment × concurrency, so a re-run replaces, never duplicates, a point, while a new server layout adds one. +- **One workload, real samples**: the additive `workloadKey` preserves the full canonical workload identity from the source, including prompts and seeds; the shortened workload label is display text only. Missing identity stays null and is isolated by original source manifest. Runtime revision remains a disclosed comparison variable. `dashboardCells` keeps only the workload measured on the most hardware and drops cells with fewer than ten valid clips (the P90 floor), so a smoke run of a shorter plan never becomes a point, a KPI lead or a Compare side; the Performance history list still shows it, and the Workload fact counts the hidden workloads. +- **Strings**: component-local `STRINGS = { en, zh }` via `useLocale()`; SKUs, units and identifiers stay English. +- **Native srt-slurm bundles**: the `h3-results--` artifacts of InferenceX #2916 (`evidence_kind: srt_slurm_h3_service`) feed the same projection; see [Video performance history](./video-history.md) for the contract. They record no GPU model, so a point joins the per-SKU chart, cards and Compare only when `ci.site.gpu_model` or the CI runner's name identifies the hardware; otherwise it stays in the Performance history list with an empty hardware. Invalid or missing power renders as not measured. +- **Fixture**: `cypress/fixtures/api/video-history.json` is rebuilt from retained `StoredArtifact` exports with `bun run fixtures:video-history ` (`scripts/build-video-history-fixture.ts`); it is metadata only and registered in `_manifest.json`. Exports for runs the fixture does not list yet are appended as new entries with an unknown publication time. A downloaded `h3-results--` bundle becomes such an export with `bun scripts/export-stored-artifact.ts `; the retained native H200 run 39002901777 entered the fixture this way. + +## Serving outcome accounting + +The serving evidence table and public API `serving` rows retain every planned cell in the loaded history pages before chart, workload, hardware or quality filters. They include zero-sample cells and source errors. Counts distinguish scheduled, attempted, completed, failed attempts, timed out (a subset of failed attempts), not started, valid and unjudged. The old failed-slot total is retained separately as `legacyFailedSlots`; it must not be read as attempted failures. Unknown evidence stays null. + +A complete request ledger and absent quality sidecar establish valid outputs as unjudged. Aggregate quality labels alone do not establish a clip-level partition. `qualitySloGoodput` stays null until media-bound quality judgments and delivery-deadline evidence can be joined for each request. This table accounts for outcomes; it does not qualify production serving or perceptual quality. + +## 中文说明 + +质量视图通过 `v_y=quality` 选择,`v_quality` 指定提示词遵循度、视觉保真度、时序一致性、运动合理性、音频质量、音频内容或音画同步,七项独立保留越高越好的 0–4 级评分,不合成为总分。七项关键维度均须有完整、已校准且通过冻结规则的证据;所选维度仅决定纵轴数值或额外筛选阈值。旧 1–5 量尺或缺少量尺声明的记录不会被转换或准入。A/A 只检查数值重复性,不能确定感知质量的可接受标准。`v_qmin` 可在计算 Pareto 前增加读者阈值,但不能放宽已冻结的协议规则。原默认 P90 对每美元视频数保持不变;保留部署预览显式选择每 GPU 小时视频数。 + +质量入选要求包括逐维度通过判定、完整样本覆盖、协议/rubric/evaluator 身份及哈希、已校准阈值与来源。Goal 1 v0 尚无已完成人工评分或校准门槛,因此当前质量视图为空;技术完整性、PSNR 和音频频谱相似度不能替代质量评价。原始已评但未获资格的分数可以保留在元数据里,不能进入质量前沿。 + +部署身份扩展到已记录的副本数、每副本 GPU 数、Ring/CFG、offload、batch、调度、engine、precision、加速方法及生成配置,缺失值保持 null。只有相同 workload 和兼容生成配置内的实测点可以比较,质量模式另按协议及 evaluator 分组。硬件健康失败和未知 batch 容量排除出正常排名;单点保留为散点,连线只连接已测前沿点。开发环境可以读取 H200/B200 2/4/8-GPU 保留数据,原回归 fixture 保持独立。 + +`/video` 以跨硬件图表为首屏:数据来自已发布 blob 索引的 per-cell 投影,浏览器不再为图表解析产物包;已发布结果的“性能历史”列表收进可展开区块,`history-*` 深链接会自动展开;原有的按运行查看、trade-off 图表与 CI 运行浏览已移除。仪表板只取硬件覆盖最广的那个工作负载,并剔除有效视频不足 10 条的 cell(与 P90 门槛一致),因此较短 plan 的 smoke 运行不会成为图上的点、KPI 领先部署或 Compare 的一方,但仍会出现在性能历史列表中;工作负载一栏会注明隐藏的其他工作负载数量。`video-url-state.ts` 管理 `v_x`、`v_y`、`v_tier`、`v_view`、`v_optimal`、`v_api`、`v_hidden` 等 URL 参数,默认值不写入 URL。页面外观与 `/inference` 对齐:标题、说明和分享按钮放在同一张 Card 的 DashboardSectionHeader 中,两组配置面板位于其下;成本档位改为在图表说明行内选择;Compare 与性能测量证据两个区块默认折叠,携带 `v_base`/`v_cand`/`v_case` 的深链接会自动展开 Compare;分享按钮会带上当前的 `v_` 参数。`v_hidden` 保存隐藏的硬件,作用于图表、表格和 CSV;KPI 卡、Compare 和证据面板仍展示所选工作负载。X 轴可选 P90(默认)或 P50 出片时间;Y 轴可选每 1 美元 TCO 视频数(默认,Hyperscaler 自有设备档位,单价与 `/inference` 同源)、每条视频 TCO 成本、每 GPU 小时视频数和每条视频 GPU 板卡能耗(kJ)。所有按 GPU 计的指标都以参与计算的 GPU 为分母(例如历史 fixture 每条视频 4 张,TP2 × Ulysses 2),而不是任务预留的板卡数;保留数据中 H100 与 B200 的运行预留了 8 张却只用 4 张,图表标题下方的说明会点名这些硬件,对应 KPI 卡标题显示“4 / 8 张 GPU”。空闲板卡只作说明、不计成本,没有 GPU 口径切换。两条坐标轴都从 0 起,便于按长度比较。 + +曲线来自部署而不是并发:保留的 fixture 使用 batch-one、单副本 H3 服务,C1/C2/C4 的吞吐量相差不到 1.5%,延迟随 C 增长;这只描述该次配置,不代表所有 diffusion 服务或未来 runtime。仪表板将旧 batch-one 记录中超过副本数的并发视为同一部署的排队观测,不把这些点连成部署曲线。真正在延迟与效率之间取舍的是部署方式:每条视频占用多少张 GPU 以及模型如何切分(`tp_size` × `ulysses_degree`)。`deployment.ts` 使用上述完整部署身份;旧 batch-one 记录仍按 `concurrency > replicas` 判断排队,现代记录按已知容量判断,并为每种硬件选出最高效的部署作为代表(KPI 卡、Compare 面板和图例都用它);`plot.ts` 只用未排队的 cell 绘图——排队 cell 不会进入图表、表格和 CSV,只出现在证据面板——再按硬件对剩余部署点求 Pareto 前沿:某硬件有两个及以上前沿部署时自动画出实线,被支配的部署以淡色显示。唯一的开关是图例中的“仅最优”(`v_optimal`,默认开启,与 inference 页的 `i_optimal` 一致):开启时图表、表格和 CSV 只包含前沿部署,关闭后被支配的部署以淡色加入图表;没有“显示排队请求”开关,也没有跨硬件前沿。保留的仪表板 fixture 中,每种已测硬件只有一种实测部署(每条视频 4 张 GPU,TP2 × Ulysses 2),所以图表是诚实的三点散点图,说明文字会指出每硬件曲线还需扫描每条视频占用的 GPU 数;该扫描属于 InferenceX 后端工作并需要 GPU 授权,前端按已记录的完整部署和工作负载身份接收新 cell。MI355X 以“未测得”形式保留在图例与 KPI 卡中;缺失指标一律为 null,绝不写成 0。 + +图表下方还有三个区块。带日期、可编辑的 API 参考价(`v_api`,记录 MiniMax 开放平台按量付费页在 2026-09-24 的价格,页面未标注生效日期;不保证当前仍适用)只推导一个数字——每条视频 API 标价(单价 × 视频秒数),显示在 KPI 卡每条视频 TCO 成本旁边,并作为表格和 CSV 的一列;它不进入图表,也不再推导收入、利润或价格倍数。价格缺失或不为正时该值为 null,`parseApiPrice` 会回退到参考价,而不是按 0 计价。Arena 式 Compare 以视频为主:同一 prompt 与 seed 在两种硬件上的视频并排播放,支持同步播放/暂停/重播和上一个/下一个用例箭头,没有盲测模式,也没有用例下拉框;下方的差值行固定为出片时间(P50)、每条视频 TCO 成本和每条视频能耗,并以脚注注明成本档位。每种硬件只有一个可选 cell(其代表部署),默认以最慢的硬件为基线、最快的为候选,选择记录在 `v_base`、`v_cand`、`v_case` 中;面板滚入视口时才加载两个已发布产物,HTTP 204 读作“未发布视频”而不是错误。性能测量证据面板展示板卡功率相对生效上限的比值、并发下的吞吐量和延迟比值,以及带宽/FLOPS 规格比与实测加速比。跨硬件比较要求完整且相同的工作负载身份、完整部署布局和 C1;缺少匹配布局的硬件直接省略,不回退到其代表部署。并发比较还要求相同副本数。文案只描述数值,不据此断言功率饱和、吞吐量平台或计算/内存瓶颈。 + +InferenceX #2916 的原生 srt-slurm 产物包(`h3-results--`,`evidence_kind: srt_slurm_h3_service`)走同一条投影路径,契约见[视频性能历史](./video-history.md)。它们不记录 GPU 型号:只有在 `ci.site.gpu_model` 或 CI runner 名称能识别出硬件时,对应点才会进入按 SKU 划分的图表、KPI 卡和 Compare;否则只保留在性能历史列表中,硬件留空。功率无效或缺失时显示为未测得。 + +完整工作负载身份由附加字段 `workloadKey` 保存,包含 prompts 与 seeds;缩写标签只用于显示。身份缺失时保持 null,按原始 source manifest 隔离,不跨来源合并。runtime revision 作为比较变量单独披露。仪表板最多读取五页历史;公开 `/api/v1/views/video` 默认复用相同选择与计算逻辑,旧 API 视图仍可显式指定。 + +`cypress/fixtures/api/video-history.json` 由保留的 `StoredArtifact` 导出通过 `bun run fixtures:video-history ` 重建,只含元数据,并登记在 `_manifest.json` 中。fixture 尚未收录的 run 会作为新条目追加在末尾,发布时间记为未知。下载的 `h3-results--` 产物包可用 `bun scripts/export-stored-artifact.ts ` 转成这种导出;保留的原生 H200 run 39002901777 就是这样进入 fixture 的。 + +请求结果表与公开 API 的 `serving` 行在图表、工作负载、硬件和质量筛选之前,保留已加载历史页中的每个计划 cell,包括零样本 cell 和来源错误。计划、尝试、完成、尝试后失败、超时(尝试后失败的子集)、未启动、有效和未评判分别计数;旧失败 slot 总数单独保存为 `legacyFailedSlots`,不能当作尝试后失败。未知证据保持 null。只有完整请求记录且没有质量 sidecar 时,才能把有效输出记为未评判;聚合标签不能推导逐视频分区。质量与截止时间证据尚未逐请求关联时,`qualitySloGoodput` 保持 null。 + +### Local multi-model foundation + +`/video?v_model=wan22` and `/api/v1/views/video?v_model=wan22` select Wan2.2-T2V-A14B; +`h3` is the default. Model filtering precedes workload selection and applies to charts, +tables, exports, serving evidence and history provenance. Switching models resets custom +pricing, hidden hardware and comparison/history selections. H3 retains its dated $0.08/video-second +reference; Wan's API price is null unless the reader supplies `v_api`. Missing results remain +empty. The serving reader accepts `video-serving--` artifacts with +`video_serving_smoke_matrix`, `controlled_video_gpu` and `live_video` evidence while preserving +legacy H3 contracts. Native selected-plan bundles use the legacy H3 kind constants for either model; generic supervised Wan bundles use the generic tags. `operator_endpoint` request records remain valid under either model's verified supervisor. Wan request duration is `frame_count / fps`; H3 keeps its explicitly requested duration. Wan's canonical model ID is `Wan-AI/Wan2.2-T2V-A14B-Diffusers`; +the previous `Wan-AI/Wan2.2-T2V-A14B` remains a selection alias, not permission to pool unequal workload keys. +`manifest.workload_plan.model_id` supplies source-level identity even before generation; +a contradictory optional result is rejected. API provenance returns nullable source `model`. +A sealed initial serving summary supplies planned/not-started counts; a manifest alone cannot infer them. +This reader does not itself establish GPU measurements, quality qualification or Trainium support. +Only legacy `h3-*` artifacts without model identity use the H3 fallback; unidentified generic artifacts do not enter either model projection. +Ambiguous mixed-model sources are excluded. H3's seven-dimension quality policy is unchanged; +Wan quality and quality/SLO goodput stay null until a separate video-only quality contract is supported. + +Wan serving 接入沿用既有目录和校验流程,接受 `video-serving--`、 +`video_serving_smoke_matrix`、`controlled_video_gpu` 和 `live_video`,同时兼容 H3。原生 selected-plan 数据包在两种模型上保留 H3 类型常量;通用 Wan supervisor 数据包使用通用类型。经对应模型 supervisor 验证的请求记录仍可使用 `operator_endpoint`。Wan 的请求时长按 `frame_count / fps` 计算,H3 保留显式请求时长。 +规范模型 ID 为 `Wan-AI/Wan2.2-T2V-A14B-Diffusers`,旧的 `Wan-AI/Wan2.2-T2V-A14B` +仅作为模型选择的别名,不会合并不同的工作负载。来源身份取自已封存的 +`manifest.workload_plan.model_id`;可选结果中的模型与之冲突时会拒绝该结果。 +公开 API 的 provenance 保留可为空的来源 `model`。只有旧 `h3-*` 产物可在模型身份缺失时归入 H3;未知模型的通用产物不会进入任一模型投影。预检失败的计划数与未启动数 +必须由初始 serving summary 明确记录,不能只凭 manifest 推导。接入成功不代表 +已有 GPU 测量、通过质量验收或支持 Trainium;Wan 的质量与质量/SLO goodput 仍为 null。 + +### Video engine identity + +Published history exposes `producer` when the manifest, matrix and execution records agree on the selected `plan_sha256`, model revision and `engine: {name, revision}`. Legacy sources retain null identity. The dashboard and API expose `releaseCoverage` for H3/Wan × SGLang/vLLM-Omni across the loaded published pages, including missing, failed, history-only and sample-floor-met states. This bounded coverage is not a full-catalog or quality qualification claim. Native four-clip runs remain History-only; the chart, KPI and Compare sample floor stays ten. + +Compare keeps distinct engine revisions on the same GPU. `v_base` and `v_cand` accept `hardware:engine:revision`; hardware-only links remain valid. Technical deltas require the same sealed model revision, selected plan and generation settings. Unknown quality does not suppress technical observations, but does not authorize quality-normalized claims. CSV rows include `model`, `model_revision`, `engine_revision` and `plan_sha256`. Compare reads `format=published` only. diff --git a/packages/app/component-timings.json b/packages/app/component-timings.json index 943d5cdd4..432329f6e 100644 --- a/packages/app/component-timings.json +++ b/packages/app/component-timings.json @@ -3492,273 +3492,304 @@ ] }, { - "spec": "cypress/component/video-ci-runs.cy.tsx", - "duration": 5955, + "spec": "cypress/component/video-api-reference.cy.tsx", + "duration": 1120, "tests": [ { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "keeps an open long-label menu within mobile bounds and supports keyboard selection" + "VideoApiReference", + "shows the dated reference with its source and applies valid prices as typed" ], - "duration": 328, + "duration": 335, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "loads the newest run automatically and switches to a failed run without inventing media" + "VideoApiReference", + "keeps the last valid price while the box is empty or zero and snaps back on blur" ], - "duration": 447, + "duration": 273, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "starts shared-artifact loading before run metadata finishes" + "VideoApiReference", + "resets to the reference, adopts external values and tracks committed changes" ], - "duration": 69, + "duration": 440, "attempts": 1 }, + { + "title": ["VideoApiReference", "renders Chinese copy under /zh"], + "duration": 53, + "attempts": 1 + } + ] + }, + { + "spec": "cypress/component/video-compare.cy.tsx", + "duration": 1803, + "tests": [ { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "keeps navigation and bundle validation errors visible on an empty tradeoff link" + "Video compare panel (retained fixture)", + "plays the same case from the slowest and fastest hardware side by side by default" ], - "duration": 158, + "duration": 216, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "keeps the shared run selected when browsing the first catalog page" + "Video compare panel (retained fixture)", + "steps through cases with the arrows, keeping v_case in the URL" ], - "duration": 358, + "duration": 365, "attempts": 1 }, { - "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "keeps a direct selection and its download active when an older catalog arrives" - ], - "duration": 3387, + "title": ["Video compare panel (retained fixture)", "drives both players together"], + "duration": 301, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "shows stored measurements without downloading the archive" + "Video compare panel (retained fixture)", + "switches the candidate, fetching only the new side, and swaps on collision or request" ], - "duration": 116, + "duration": 675, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "opens the shared serving cell and switches from the chart without reloading the CI artifact" + "Video compare panel (retained fixture)", + "restores v_base and v_cand from the URL and reports a failed media read with a retry" ], - "duration": 760, + "duration": 57, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "shows a malformed serving matrix error instead of empty paired results" + "Video compare panel (retained fixture)", + "says when a run publishes no clips and keeps the metric deltas" ], - "duration": 44, + "duration": 50, "attempts": 1 }, { "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "reports expiration without downloading or substituting another artifact" + "Video compare panel (retained fixture)", + "disables the selects when fewer than two hardware are measured" ], - "duration": 34, + "duration": 30, "attempts": 1 }, { - "title": [ - "H3 automatic CI viewer (synthetic API fixtures)", - "withholds invalid generation measurements and separates warmup energy" - ], - "duration": 207, + "title": ["Video compare panel (retained fixture)", "renders Chinese copy on /zh/video"], + "duration": 86, "attempts": 1 } ] }, { - "spec": "cypress/component/video-fidelity.cy.tsx", - "duration": 1069, + "spec": "cypress/component/video-dashboard.cy.tsx", + "duration": 3618, "tests": [ { "title": [ - "Native paired fidelity UI (synthetic artifacts)", - "distinguishes valid media from 18 threshold failures and changes both selected videos" + "Video hardware dashboard (retained fixture)", + "plots one C1 point per measured hardware and lists MI355X as unavailable" ], - "duration": 498, + "duration": 124, "attempts": 1 }, { "title": [ - "Native paired fidelity UI (synthetic artifacts)", - "rewrites report media/download links and removes scripts and external resources" + "Video hardware dashboard (retained fixture)", + "states model, workload and deployment as facts; the legend offers only Optimal Only" ], - "duration": 226, + "duration": 73, "attempts": 1 }, { "title": [ - "Native paired fidelity UI (synthetic artifacts)", - "loads a raw sealed bundle and revokes owned object URLs when unmounted" + "Video hardware dashboard (retained fixture)", + "switches the cost tier, repricing the cards, the badges and the URL" ], - "duration": 94, + "duration": 245, "attempts": 1 }, { "title": [ - "Native paired fidelity UI (synthetic artifacts)", - "clears old evidence and reports an invalid or empty comparison" + "Video hardware dashboard (retained fixture)", + "never plots queued cells, even when a stale URL asks for them" ], - "duration": 105, + "duration": 240, "attempts": 1 }, { "title": [ - "Native paired fidelity UI (synthetic artifacts)", - "renders Chinese status and original-audio controls without relabeling failures as invalid media" + "Video hardware dashboard (retained fixture)", + "lists the plotted cells with the full column set on demand and no concurrency column" ], - "duration": 122, + "duration": 150, "attempts": 1 - } - ] - }, - { - "spec": "cypress/component/video-serving-results.cy.tsx", - "duration": 2301, - "tests": [ + }, { "title": [ - "H3 serving results (synthetic fixtures)", - "follows chart-driven cell changes while the result remains mounted" + "Video hardware dashboard (retained fixture)", + "hides a hardware from the legend and shows the table view" ], - "duration": 567, + "duration": 287, "attempts": 1 }, { "title": [ - "H3 serving results (synthetic fixtures)", - "renders a parent-created blob in the report while keeping scripts disabled" + "Video hardware dashboard (retained fixture)", + "restores hidden hardware from a shared table URL" ], - "duration": 242, + "duration": 140, "attempts": 1 }, { "title": [ - "H3 serving results (synthetic fixtures)", - "shows each concurrency without manufacturing a candidate and separates participating and allocated GPU counts" + "Video hardware dashboard (retained fixture)", + "restores v_ params from the URL and opens the history section for history deep links" ], - "duration": 646, + "duration": 73, "attempts": 1 }, { "title": [ - "H3 serving results (synthetic fixtures)", - "shows server stages separately from delivery and retains partial coverage" + "Video hardware dashboard (retained fixture)", + "reprices the API list price beside the TCO cost, in the cards, the table and the URL" ], - "duration": 223, + "duration": 629, "attempts": 1 }, { "title": [ - "H3 serving results (synthetic fixtures)", - "distinguishes requested and decoded duration and withholds invalid power" + "Video hardware dashboard (retained fixture)", + "toggles Optimal Only from the legend and restores it from v_optimal" ], - "duration": 267, + "duration": 253, "attempts": 1 }, { "title": [ - "H3 serving results (synthetic fixtures)", - "withholds headline metrics and power for an unverified configuration" + "Video hardware dashboard (retained fixture)", + "folds Compare and the evidence by default and opens Compare for compare deep links" ], - "duration": 61, + "duration": 253, "attempts": 1 }, { "title": [ - "H3 serving results (synthetic fixtures)", - "supports Chinese copy and missing request, media, and allocation data" + "Video hardware dashboard (retained fixture)", + "shows placeholders while loading instead of a false \"Not measured\"" ], - "duration": 264, + "duration": 872, + "attempts": 1 + }, + { + "title": [ + "Video hardware dashboard (retained fixture)", + "renders Chinese copy on /zh/video" + ], + "duration": 229, "attempts": 1 } ] }, { - "spec": "cypress/component/video-tradeoff.cy.tsx", - "duration": 4419, + "spec": "cypress/component/video-evidence.cy.tsx", + "duration": 607, "tests": [ { "title": [ - "Video tradeoff chart (synthetic fixtures)", - "plots participating GPU efficiency by default and requires assumptions for the cost axis" + "Video performance evidence (retained fixture)", + "keeps the concurrency table within the stated shared layout (en, 1280px)" ], - "duration": 2525, + "duration": 150, "attempts": 1 }, { "title": [ - "Video tradeoff chart (synthetic fixtures)", - "still withholds P90 for single-request results until median diagnostics are selected" + "Video performance evidence (retained fixture)", + "keeps the concurrency table within the stated shared layout (zh, 390px)" ], - "duration": 170, + "duration": 137, "attempts": 1 }, { "title": [ - "Video tradeoff chart (synthetic fixtures)", - "shows an empty state without inventing fixture results" + "Video performance evidence (retained fixture)", + "does not call low board power compute saturation" ], - "duration": 39, + "duration": 26, "attempts": 1 }, { "title": [ - "Video tradeoff chart (synthetic fixtures)", - "distinguishes groups with the same short label and exposes their full workload" + "Video performance evidence (retained fixture)", + "does not call higher throughput and lower latency a queueing plateau" ], - "duration": 496, + "duration": 25, "attempts": 1 }, { "title": [ - "Video serving matrix chart (synthetic contract fixture)", - "shows server coverage in point details without changing the client latency axis" + "Video performance evidence (retained fixture)", + "reads board power against the recorded enforced limit per measured hardware" ], - "duration": 187, + "duration": 31, "attempts": 1 }, { "title": [ - "Video serving matrix chart (synthetic contract fixture)", - "defaults to P90 when individual serving cells have enough samples" + "Video performance evidence (retained fixture)", + "bounds the power meter to its declared range when mean power exceeds the recorded limit" ], - "duration": 78, + "duration": 27, "attempts": 1 }, { "title": [ - "Video serving matrix chart (synthetic contract fixture)", - "shows three labeled cell medians and opens the selected concurrency result" + "Video performance evidence (retained fixture)", + "tabulates concurrency measurements with ratios against each hardware C1 cell" ], - "duration": 754, + "duration": 36, "attempts": 1 }, { "title": [ - "Video serving matrix chart (synthetic contract fixture)", - "shows the Chinese serving labels with per-cell medians" + "Video performance evidence (retained fixture)", + "compares observed speedups with spec-sheet ratios without attributing a bottleneck" ], - "duration": 120, + "duration": 31, + "attempts": 1 + }, + { + "title": [ + "Video performance evidence (retained fixture)", + "renders Chinese copy on /zh/video at mobile width" + ], + "duration": 102, + "attempts": 1 + }, + { + "title": [ + "Video performance evidence (retained fixture)", + "hides the plateau and scaling exhibits when only one hardware at C1 is measured" + ], + "duration": 28, + "attempts": 1 + }, + { + "title": [ + "Video performance evidence (retained fixture)", + "renders nothing when no cell is measured" + ], + "duration": 6, "attempts": 1 } ] diff --git a/packages/app/cypress/component/video-api-reference.cy.tsx b/packages/app/cypress/component/video-api-reference.cy.tsx new file mode 100644 index 000000000..e177409b7 --- /dev/null +++ b/packages/app/cypress/component/video-api-reference.cy.tsx @@ -0,0 +1,78 @@ +import { useState } from 'react'; +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; +import VideoApiReference from '@/components/video-benchmark/VideoApiReference'; +import { registerAnalyticsClient } from '@/lib/analytics/analytics'; + +/** Owns the price like VideoDashboard does and exposes it, plus an external setter. */ +function Harness({ initial, pathname = '/video' }: { initial: number; pathname?: string }) { + const [value, setValue] = useState(initial); + return ( + +
+ +
+ {value} + +
+ ); +} +const input = () => cy.get('[data-testid="video-api-price"]'); +const value = () => cy.get('[data-testid="api-price-value"]'); +const caption = () => cy.get('[data-testid="video-api-reference-caption"]'); + +describe('VideoApiReference', () => { + it('shows the dated reference with its source and applies valid prices as typed', () => { + cy.mount(); + input().should('have.value', '0.08').and('have.attr', 'step', '0.001'); + caption() + .should('contain', 'Reference $0.080/video-s') + .and('contain', 'listed range $0.080–$0.130') + .and('contain', 'captured 2026-09-24') + .and('contain', 'Source: MiniMax platform pay-as-you-go pricing page') + .and('contain', 'page shows no effective date') + .and('contain', 'List price, not realized revenue'); + input().clear().type('0.05'); + value().should('have.text', '0.05'); + }); + it('keeps the last valid price while the box is empty or zero and snaps back on blur', () => { + cy.mount(); + input().clear(); + value().should('have.text', '0.08'); + input().type('0'); + value().should('have.text', '0.08'); + input().blur().should('have.value', '0.08'); + }); + it('resets to the reference, adopts external values and tracks committed changes', () => { + const capture = cy.stub().as('capture'); + registerAnalyticsClient({ capture }); + cy.mount(); + input().should('have.value', '0.05'); + cy.get('[data-testid="video-api-price-reset"]').click(); + value().should('have.text', '0.08'); + input().should('have.value', '0.08'); + cy.get('@capture').should('have.been.calledWith', 'video_api_price_changed', { + value: '0.08', + reset: true, + }); + cy.get('[data-testid="api-price-external"]').click(); + input().should('have.value', '0.047'); + input().clear().type('0.04').blur(); + value().should('have.text', '0.04'); + cy.get('@capture').should('have.been.calledWith', 'video_api_price_changed', { + value: '0.04', + }); + }); + it('renders Chinese copy under /zh', () => { + cy.mount(); + cy.contains('label', 'API 参考价($/video-s)').should('exist'); + cy.contains('button', '重置').should('exist'); + caption() + .should('contain', '参考值 $0.080/video-s') + .and('contain', '标价区间 $0.080–$0.130') + .and('contain', '采集于 2026-09-24') + .and('contain', '来源:MiniMax 开放平台按量付费价格页') + .and('contain', '此处为 API 标价,而非实际收入'); + }); +}); diff --git a/packages/app/cypress/component/video-ci-runs.cy.tsx b/packages/app/cypress/component/video-ci-runs.cy.tsx deleted file mode 100644 index aaf4429ac..000000000 --- a/packages/app/cypress/component/video-ci-runs.cy.tsx +++ /dev/null @@ -1,444 +0,0 @@ -import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; -import VideoCIRuns from '@/components/video-benchmark/VideoCIRuns'; -import ResultPower from '@/components/video-benchmark/ResultPower'; -import VideoSelect from '@/components/video-benchmark/VideoSelect'; -import { servingArtifact, videoRun as run } from '../support/video-artifacts'; - -describe('H3 automatic CI viewer (synthetic API fixtures)', () => { - beforeEach(() => { - cy.intercept('GET', '**/api/video-runs*format=media', { statusCode: 204 }); - cy.window().then((win) => win.history.replaceState(null, '', win.location.pathname)); - }); - it('keeps an open long-label menu within mobile bounds and supports keyboard selection', () => { - cy.viewport(390, 844); - const change = cy.stub().as('selection'); - cy.mount( -
- -
, - ); - cy.get('[role="combobox"]').focus().type('{enter}'); - cy.get('[role="listbox"]') - .should('be.visible') - .then(($menu) => { - const rect = $menu[0].getBoundingClientRect(); - expect(rect.left).to.be.at.least(0); - expect(rect.right).to.be.at.most(390); - }); - cy.get('[role="option"]').last().focus().type('{enter}'); - cy.get('@selection').should('have.been.calledWith', 'second'); - cy.get('[role="listbox"]').should('not.exist'); - cy.get('[role="combobox"]').should('have.focus'); - }); - it('loads the newest run automatically and switches to a failed run without inventing media', () => { - cy.intercept('GET', '/api/video-runs?page=1', { - runs: [run(20, 'success'), run(10, 'failure')], - nextPage: null, - }); - cy.intercept('GET', '/api/video-runs?run=20', { run: run(20, 'success'), artifacts: [] }).as( - 'latest', - ); - cy.intercept('GET', '/api/video-runs?run=10', { run: run(10, 'failure'), artifacts: [] }).as( - 'failed', - ); - cy.mount( - - - , - ); - cy.wait('@latest'); - cy.contains('No result artifact for this run yet').should('be.visible'); - cy.get('[role="combobox"][aria-label="CI run"]').click(); - cy.contains('[role="option"]', '#10 · failure').click(); - cy.wait('@failed'); - cy.contains('summary', 'Run details and artifact selection').click(); - cy.contains('a', '#10 · completed / failure').should('be.visible'); - cy.get('video').should('not.exist'); - }); - it('starts shared-artifact loading before run metadata finishes', () => { - let releaseRun: (() => void) | undefined; - let mediaStarted = false; - cy.window().then((win) => - win.history.replaceState(null, '', `${win.location.pathname}?run=30&artifact=40`), - ); - cy.intercept( - 'GET', - '/api/video-runs?run=30', - (req) => - new Promise((resolve) => { - releaseRun = () => { - req.reply({ - run: run(30, 'success'), - artifacts: [{ id: 40, name: 'h3-results-30-1', expired: false, size_in_bytes: 100 }], - }); - resolve(); - }; - }), - ); - cy.intercept('GET', '**/api/video-runs*format=media', (req) => { - mediaStarted = true; - req.reply({ statusCode: 204 }); - }); - cy.intercept('GET', '/api/video-runs?run=30&artifact=40', { - statusCode: 502, - body: { error: 'Synthetic ZIP fallback' }, - }); - cy.mount( - - - , - ); - cy.wrap(null).should(() => { - expect(mediaStarted).to.equal(true); - expect(releaseRun).to.be.a('function'); - }); - cy.then(() => releaseRun?.()); - cy.get('[data-testid="video-ci-runs"] [role="alert"]').should( - 'contain', - 'Synthetic ZIP fallback', - ); - }); - it('keeps navigation and bundle validation errors visible on an empty tradeoff link', () => { - cy.window().then((win) => - win.history.replaceState( - null, - '', - `${win.location.pathname}?run=30&artifact=40&view=tradeoff`, - ), - ); - cy.intercept('GET', '/api/video-runs?run=30', { - run: run(30, 'success'), - artifacts: [{ id: 40, name: 'h3-results-30-1', expired: false, size_in_bytes: 100 }], - }); - cy.intercept('GET', '**/api/video-runs*format=media', { - storageVersion: 1, - runId: '30', - artifact: { id: 40, name: 'h3-results-30-1', expired: false, size_in_bytes: 100 }, - sources: [ - { - id: '30', - assets: [], - checksums: [['manifest.json', 'synthetic']], - texts: [['gpu/supervisor/baseline/telemetry.jsonl', 'broken JSON']], - documents: [ - ['manifest.json', { run_id: '30' }], - [ - 'gpu/gpu-job.json', - { - roles: { - baseline: { - telemetry_summary: { - qualified: true, - requested_interval_seconds: 1, - gpu_identity: [{ uuid: 'GPU-synthetic' }], - }, - }, - }, - }, - ], - ], - }, - ], - }); - cy.mount( - - - , - ); - cy.contains('button', 'Videos & result').should('be.visible'); - cy.contains('[role="alert"]', 'JSON').should('be.visible'); - cy.contains('button', 'Videos & result').click(); - cy.get('[data-testid="video-tradeoff"]').should('not.be.visible'); - cy.contains('[role="alert"]', 'JSON').should('be.visible'); - }); - it('keeps the shared run selected when browsing the first catalog page', () => { - cy.window().then((win) => - win.history.replaceState(null, '', `${win.location.pathname}?run=30`), - ); - cy.intercept('GET', '/api/video-runs?run=30', { run: run(30, 'success'), artifacts: [] }).as( - 'shared', - ); - cy.intercept('GET', '/api/video-runs?page=1', { runs: [run(20, 'success')], nextPage: 2 }).as( - 'browse', - ); - cy.mount( - - - , - ); - cy.wait('@shared'); - cy.get('[role="combobox"][aria-label="CI run"]').should('contain.text', '#30'); - cy.contains('button', 'Older runs').should('not.exist'); - cy.contains('button', 'Browse CI runs').click(); - cy.wait('@browse'); - cy.get('[role="combobox"][aria-label="CI run"]').should('contain.text', '#30').click(); - cy.contains('[role="option"]', '#20 · success').should('be.visible'); - cy.get('[role="listbox"]').type('{esc}'); - cy.contains('button', 'Older runs').should('be.visible'); - cy.location('search').should('contain', 'run=30'); - }); - it('keeps a direct selection and its download active when an older catalog arrives', () => { - let releaseList: (() => void) | undefined; - cy.intercept( - 'GET', - '/api/video-runs?page=1', - (req) => - new Promise((resolve) => { - releaseList = () => { - req.reply({ runs: [run(20, 'success')], nextPage: null }); - resolve(); - }; - }), - ).as('catalog'); - cy.intercept('GET', '/api/video-runs?run=30', { - run: run(30, 'success'), - artifacts: [{ id: 40, name: 'h3-results-30-1', expired: false, size_in_bytes: 100 }], - }).as('direct'); - cy.intercept('GET', '/api/video-runs?run=30&artifact=40', { - delay: 3000, - statusCode: 502, - body: { error: 'Synthetic download failure' }, - }).as('zip'); - cy.mount( - - - , - ); - cy.wrap(null).should(() => expect(releaseList).to.be.a('function')); - cy.contains('summary', 'Run details and artifact selection').click(); - cy.get('input[aria-label="GitHub run ID"]').type('30'); - cy.contains('button', 'Open run').click(); - cy.wait('@direct'); - cy.get('[role="combobox"][aria-label="Result artifact"]').should( - 'contain.text', - 'h3-results-30-1', - ); - cy.then(() => releaseList?.()); - cy.wait('@catalog'); - cy.get('[role="combobox"][aria-label="CI run"]').should('contain.text', '#30'); - cy.get('[role="status"]').should('be.visible'); - cy.wait('@zip'); - cy.get('[role="alert"]').should('contain', 'Synthetic download failure'); - }); - it('shows stored measurements without downloading the archive', () => { - const artifact = { - id: 40, - name: 'h3-results-30-1', - expired: false, - size_in_bytes: 100, - stored: true, - }; - cy.intercept('GET', '/api/video-runs?page=1', { runs: [run(30, 'success')], nextPage: null }); - cy.intercept('GET', '/api/video-runs?run=30', { - run: run(30, 'success'), - artifacts: [artifact], - }); - cy.intercept('GET', '/api/video-runs?run=30&artifact=40', () => { - throw new Error('Stored results must not fetch the ZIP'); - }); - cy.intercept('GET', '**/api/video-runs*format=media', { - storageVersion: 1, - runId: '30', - artifact, - sources: [ - { - id: '30', - assets: [], - texts: [], - checksums: [['manifest.json', 'a'.repeat(64)]], - documents: [ - ['manifest.json', { run_id: '30' }], - [ - 'ci.json', - { phase: 'complete', regression_status: 'inconclusive', release_qualified: false }, - ], - [ - 'report/evidence.json', - { - policy: { calibration_status: 'uncalibrated' }, - roles: { - baseline: { - summary: { latency_median_seconds: 150, valid_clips_per_second: 0.01 }, - }, - candidate: { - summary: { latency_median_seconds: 120, valid_clips_per_second: 0.02 }, - }, - }, - }, - ], - ], - }, - ], - }); - cy.mount( - - - , - ); - cy.get('[data-testid="result-summary"]') - .should('contain', 'Media stored for direct playback') - .and('contain', 'uncalibrated'); - cy.contains('tr', 'Latency') - .should('contain', '150') - .and('contain', '120') - .and('contain', '-20%'); - cy.contains('tr', 'Valid clips').should('contain', '36').and('contain', '72'); - cy.contains('tr', 'Mean GPU power').find('[aria-label="Unavailable"]').should('have.length', 2); - }); - it('opens the shared serving cell and switches from the chart without reloading the CI artifact', () => { - const saved = servingArtifact(); - cy.window().then((win) => - win.history.replaceState( - null, - '', - `${win.location.pathname}?run=123&artifact=40&source=123&cell=c4`, - ), - ); - cy.intercept('GET', '/api/video-runs?run=123', { - run: run(123, 'success'), - artifacts: [saved.artifact], - }).as('servingRun'); - cy.intercept('GET', '**/api/video-runs*format=media', saved).as('servingMedia'); - cy.intercept('GET', 'https://media.test/**', { statusCode: 204 }); - cy.intercept('GET', '/api/video-runs?run=123&artifact=40', () => { - throw new Error('Stored serving results must not fetch the ZIP'); - }); - cy.mount( - - - , - ); - cy.wait(['@servingRun', '@servingMedia']); - cy.get('[data-testid="serving-selected-metrics"]') - .should('contain', 'C4') - .and('contain', '300 s'); - cy.get('[data-testid="serving-media"] video').should( - 'have.attr', - 'src', - 'https://media.test/gpu/c4/baseline/artifacts/measurement-r001-c001.mp4', - ); - cy.get('[data-testid="serving-media"] [aria-label="Clip / request"]').click(); - cy.contains('[role="option"]', 'Warmup').click(); - cy.get('[data-testid="serving-media"] video').should( - 'have.attr', - 'src', - 'https://media.test/gpu/c4/baseline/artifacts/warmup-001.mp4', - ); - cy.contains('button', 'Hardware tradeoffs').click(); - cy.get('[data-testid="video-tradeoff"] tbody tr') - .should('have.length', 3) - .first() - .find('button') - .click(); - cy.contains('button', 'Open videos and full result').click(); - cy.get('[data-testid="serving-selected-metrics"]') - .should('be.visible') - .and('contain', 'C1') - .and('contain', '120 s'); - cy.get('[data-testid="serving-media"] video').should( - 'have.attr', - 'src', - 'https://media.test/gpu/c1/baseline/artifacts/measurement-r001-c001.mp4', - ); - cy.location('search').should('contain', 'cell=c1'); - cy.get('@servingRun.all').should('have.length', 1); - cy.get('@servingMedia.all').should('have.length', 1); - cy.get('[data-testid="result-summary"]').should('not.exist'); - }); - it('shows a malformed serving matrix error instead of empty paired results', () => { - const saved = servingArtifact(); - const source = saved.sources[0]; - source.documents = source.documents.map(([path, document]) => [ - path, - path === 'serving-smoke.json' - ? { schema_version: '9.0.0', bundle_type: 'h3_serving_smoke_matrix' } - : document, - ]); - cy.window().then((win) => - win.history.replaceState( - null, - '', - `${win.location.pathname}?run=123&artifact=40&view=tradeoff`, - ), - ); - cy.intercept('GET', '/api/video-runs?run=123', { - run: run(123, 'success'), - artifacts: [saved.artifact], - }); - cy.intercept('GET', '**/api/video-runs*format=media', saved); - cy.mount( - - - , - ); - cy.contains('[role="alert"]', 'Invalid H3 serving matrix: unsupported matrix contract').should( - 'be.visible', - ); - cy.get('[data-testid="serving-results"], [data-testid="result-summary"]').should('not.exist'); - cy.get('video').should('not.exist'); - }); - it('reports expiration without downloading or substituting another artifact', () => { - cy.intercept('GET', '/api/video-runs?page=1', { runs: [run(30, 'success')], nextPage: null }); - cy.intercept('GET', '/api/video-runs?run=30', { - run: run(30, 'success'), - artifacts: [{ id: 40, name: 'h3-results-30-1', expired: true, size_in_bytes: 100 }], - }); - cy.mount( - - - , - ); - cy.get('[role="alert"]').should('contain', 'Expired'); - cy.get('video').should('not.exist'); - cy.get('[role="combobox"][aria-label="Result artifact"]').should( - 'contain.text', - 'h3-results-30-1', - ); - }); - it('withholds invalid generation measurements and separates warmup energy', () => { - const power = { - phases: { - measurement: { - valid: false, - status: 'invalid', - invalid_reasons: ['unbracketed'], - aggregate: { avg_power_w: 999 }, - }, - warmup: { - valid: true, - status: 'valid', - duration_seconds: 2, - aggregate: { - avg_power_w: 100, - energy_j: 200, - joules_per_valid_clip: 200, - observed_peak_power_w: 110, - }, - per_gpu: { 'GPU-fixture': { avg_power_w: 100, observed_peak_power_w: 110 } }, - }, - }, - }; - cy.mount( - - - , - ); - cy.get('[data-testid="result-power"]') - .should('contain', 'unbracketed') - .and('not.contain', '999'); - cy.get('[role="combobox"][aria-label="Power window"]').click(); - cy.contains('[role="option"]', 'Warmup').click(); - cy.get('[data-testid="result-power"]') - .should('contain', '0.2') - .and('contain', 'GPU-fixture') - .and('not.contain', 'unbracketed'); - }); -}); diff --git a/packages/app/cypress/component/video-compare.cy.tsx b/packages/app/cypress/component/video-compare.cy.tsx new file mode 100644 index 000000000..442a75c32 --- /dev/null +++ b/packages/app/cypress/component/video-compare.cy.tsx @@ -0,0 +1,277 @@ +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; +import type { VideoHistoryPage } from '@/components/video-benchmark/history'; +import type { MetricOptions, VideoPoint } from '@/components/video-benchmark/metrics'; +import { dashboardCells, videoPoints } from '@/components/video-benchmark/points'; +import VideoCompare from '@/components/video-benchmark/VideoCompare'; +import history from '../fixtures/api/video-history.json'; +import { servingArtifact } from '../support/video-artifacts'; + +// Retained H100/H200/B200 C1/C2/C4 observations; clips come from synthetic stored artifacts. +const points = videoPoints([history as unknown as VideoHistoryPage]); +// The dashboard hands Compare its cells, so the retained native H200 run under the sample floor never becomes a side. +const cells = dashboardCells(points).cells; +const c1 = (key: string) => cells.find((p) => p.hardwareKey === key && p.concurrency === 1)!; +const COLORS: Record = { + h100: 'rgb(10, 20, 30)', + h200: 'rgb(40, 50, 60)', + b200: 'rgb(70, 80, 90)', +}; +const colorFor = (key: string) => COLORS[key] ?? 'rgb(0, 0, 0)'; +const OWNING: MetricOptions = { tier: 'h' }; + +const mediaUrl = (point: VideoPoint) => + `/api/video-runs?run=${point.runId}&artifact=${point.artifactId}&format=published`; +function media(point: VideoPoint, hardware: string, alias: string) { + cy.intercept( + 'GET', + mediaUrl(point), + servingArtifact(Number(point.runId), point.artifactId, hardware), + ).as(alias); +} + +/** Clips load on their own once the panel is in view, so overrides must be registered before mounting. */ +function mount( + pathname = '/video', + search = '', + data: VideoPoint[] = cells, + options: MetricOptions = OWNING, + beforeMount?: () => void, +) { + media(c1('h100'), 'NVIDIA H100 80GB HBM3', 'h100'); + media(c1('b200'), 'NVIDIA B200', 'b200'); + media(c1('h200'), 'NVIDIA H200', 'h200'); + cy.intercept('GET', 'https://media.test/**', { statusCode: 204 }); + beforeMount?.(); + cy.window().then((win) => + win.history.replaceState(null, '', `${win.location.pathname}${search}`), + ); + cy.mount( + + + , + ); +} +const table = () => cy.get('[data-testid="video-compare-table"]'); +const metricRow = (id: string) => table().find(`tbody tr[data-metric="${id}"]`); +const clip = (role: 'baseline' | 'candidate') => + cy.get(`[data-testid="video-compare-clip"][data-role="${role}"]`); +const pick = (label: string, option: string) => { + cy.get(`[role="combobox"][aria-label="${label}"]`).click(); + cy.contains('[role="option"]', option).click(); +}; + +describe('Video compare panel (retained fixture)', () => { + for (const width of [1440, 390]) { + it(`selects two engines on H200 and restores the swap at ${width}px`, () => { + cy.viewport(width, 900); + const base = c1('h200'); + const data: VideoPoint[] = ['sglang', 'vllm-omni'].map((engine, index) => ({ + ...base, + id: `${base.id}-${engine}`, + p50: 100 - index * 20, + producer: { + modelRevision: 'a'.repeat(40), + planSha256: 'b'.repeat(64), + engine: { + name: engine === 'sglang' ? 'sglang' : 'vllm-omni', + revision: String(index + 1).repeat(40), + }, + }, + })); + mount('/video', '', data); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'sglang'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'vllm-omni'); + pick('Baseline', 'vllm-omni'); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'vllm-omni'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'sglang'); + cy.location('search').should('contain', 'vllm-omni'); + cy.document().then((doc) => expect(doc.documentElement.scrollWidth).to.be.at.most(width)); + cy.screenshot(`video-engine-compare-${width}`); + }); + } + it('plays the same case from the slowest and fastest hardware side by side by default', () => { + mount(); + cy.wait(['@h100', '@b200']); + cy.get('[data-testid="video-compare-prompt"]') + .should('contain', 'A drummer taps a snare drum.') + .and('contain', 'seed 11'); + cy.get('[data-testid="video-compare-case-of"]').should('contain', 'Case 1 of'); + clip('baseline') + .should('contain', 'H100') + .and('contain', '4 GPU · TP2 × Ulysses 2') + .and('contain', 'time to video 120 s') + .find('video') + .should('have.attr', 'src') + .and('include', 'measurement-r001-c001.mp4'); + clip('candidate').should('contain', 'B200').find('video').should('exist'); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'H100'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'B200'); + table() + .find('tbody tr') + .should(($rows) => { + expect([...$rows].map((el) => el.dataset.metric)).to.deep.equal([ + 'p50Latency', + 'dollarsPerVideo', + 'kjPerVideo', + ]); + }); + metricRow('p50Latency') + .should('contain', '167.3') + .and('contain', '77.9') + .and('contain', '-53.4%'); + metricRow('p50Latency').find('[data-tone="better"]').should('exist'); + // $1.17 ÷ 5.3741 videos/GPU-hr vs. $1.73 ÷ 11.526, both per participating GPU. + metricRow('dollarsPerVideo') + .should('contain', '$0.218') + .and('contain', '$0.150') + .and('contain', '-31.1%'); + metricRow('kjPerVideo') + .should('contain', '431.1') + .and('contain', '301.1') + .and('contain', '-30.2%'); + metricRow('kjPerVideo').find('[data-tone="better"]').should('exist'); + cy.contains('[data-testid="video-compare"] p', 'Cost tier:').should( + 'have.text', + 'Cost tier: Owning at Large Hyperscaler Volume.', + ); + cy.get('[data-testid="video-compare"]') + .should('contain', 'retained CI outputs') + .and('not.contain', 'GPU basis'); + cy.get('[data-testid="video-compare-blind"]').should('not.exist'); + cy.get('[data-testid="video-compare-controls"] [role="combobox"]').should(($boxes) => { + expect([...$boxes].map((el) => el.getAttribute('aria-label'))).to.deep.equal([ + 'Baseline', + 'Candidate', + ]); + }); + cy.get('@h200.all').should('have.length', 0); + }); + it('steps through cases with the arrows, keeping v_case in the URL', () => { + mount(); + cy.wait(['@h100', '@b200']); + cy.get('[data-testid="video-compare-prev"]').should('be.disabled'); + cy.get('[role="combobox"][aria-label="Case (prompt · seed)"]').should('not.exist'); + cy.get('[data-testid="video-compare-next"]').click(); + cy.get('[data-testid="video-compare-clip"] video').each(($video) => + expect($video.attr('src')).to.include('measurement-r002-c001.mp4'), + ); + cy.get('[data-testid="video-compare-case-of"]').should('contain', 'Case 2 of'); + cy.location('search').should('contain', 'v_case=1'); + cy.get('[data-testid="video-compare-next"]').click(); + cy.get('[data-testid="video-compare-clip"] video').each(($video) => + expect($video.attr('src')).to.include('measurement-r003-c001.mp4'), + ); + cy.get('[data-testid="video-compare-case-of"]').should('contain', 'Case 3 of'); + cy.location('search').should('contain', 'v_case=2'); + cy.get('[data-testid="video-compare-prev"]').click(); + cy.get('[data-testid="video-compare-case-of"]').should('contain', 'Case 2 of'); + cy.location('search').should('contain', 'v_case=1'); + }); + it('drives both players together', () => { + mount(); + cy.wait(['@h100', '@b200']); + cy.window().then((win) => { + cy.stub(win.HTMLMediaElement.prototype, 'play').as('play').resolves(); + cy.stub(win.HTMLMediaElement.prototype, 'pause').as('pause'); + }); + cy.get('[data-testid="video-compare-play"]').click(); + cy.get('@play').should('have.callCount', 2); + cy.get('[data-testid="video-compare-pause"]').click(); + cy.get('@pause').should('have.callCount', 2); + cy.get('[data-testid="video-compare-restart"]').click(); + cy.get('@play').should('have.callCount', 4); + }); + it('switches the candidate, fetching only the new side, and swaps on collision or request', () => { + mount(); + cy.wait(['@h100', '@b200']); + pick('Candidate', 'H200'); + cy.wait('@h200'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H200'); + clip('candidate').should('contain', 'H200'); + metricRow('p50Latency').should('contain', '150.6').and('contain', '-10%'); + metricRow('dollarsPerVideo').should('contain', '$0.204').and('contain', '-6.2%'); + metricRow('kjPerVideo').should('contain', '411').and('contain', '-4.7%'); + cy.location('search').should('contain', 'v_cand=h200'); + cy.get('@h100.all').should('have.length', 1); + cy.get('@h200.all').should('have.length', 1); + // Choosing the baseline's hardware as candidate swaps the two sides; the swap button does too. + pick('Candidate', 'H100'); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'H200'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H100'); + metricRow('p50Latency').should('contain', '+11.1%').find('[data-tone="worse"]').should('exist'); + cy.get('[data-testid="video-compare-swap"]').click(); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'H100'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H200'); + cy.get('@h200.all').should('have.length', 1); + }); + it('restores v_base and v_cand from the URL and reports a failed media read with a retry', () => { + mount('/video', '?v_base=h200&v_cand=h100', cells, OWNING, () => { + cy.intercept('GET', mediaUrl(c1('h200')), { statusCode: 404, body: { error: 'gone' } }).as( + 'missing', + ); + }); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'H200'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H100'); + cy.wait('@missing'); + cy.get('[data-testid="video-compare-clips"] [role="alert"]') + .should('contain', 'Could not load clips') + .and('contain', 'HTTP 404'); + cy.contains('button', 'Retry').should('exist'); + }); + it('keeps a restored, repaired pair when one side changes', () => { + // v_cand on the default baseline makes restore swap the baseline to B200; that repair must + // stay explicit, so a later candidate change cannot re-default the baseline to H100. + mount('/video', '?v_cand=h100'); + cy.wait(['@b200', '@h100']); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'B200'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H100'); + pick('Candidate', 'H200'); + cy.wait('@h200'); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'B200'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H200'); + cy.location('search').should('contain', 'v_base=b200').and('contain', 'v_cand=h200'); + }); + it('says when a run publishes no clips and keeps the metric deltas', () => { + mount('/video', '', cells, OWNING, () => { + cy.intercept('GET', mediaUrl(c1('b200')), { statusCode: 204 }).as('unpublished'); + }); + cy.wait(['@h100', '@unpublished']); + cy.get('[data-testid="video-compare-clips"]').should( + 'contain', + 'Clips are not published for one of these runs', + ); + cy.get('[data-testid="video-compare-clip"]').should('not.exist'); + metricRow('p50Latency').should('contain', '-53.4%'); + }); + it('disables the selects when fewer than two hardware are measured', () => { + mount( + '/video', + '', + cells.filter((p) => p.hardwareKey === 'h200'), + ); + cy.get('[data-testid="video-compare-controls"]').should('have.attr', 'disabled'); + cy.get('[data-testid="video-compare"]').should( + 'contain', + 'Comparison needs at least two measured hardware or engine configurations.', + ); + cy.get('[data-testid="video-compare-table"]').should('not.exist'); + }); + it('renders Chinese copy on /zh/video', () => { + mount('/zh/video', '', cells, { tier: 'r' }); + cy.wait(['@h100', '@b200']); + cy.get('[data-testid="video-compare"]') + .should('contain', '基线') + .and('contain', '候选') + .and('contain', '同时播放') + .and('contain', '成本档位:租赁 - 3 年承诺。'); + metricRow('p50Latency').should('contain', 'P50 出片时间'); + // Rent - 3 Year Commit: $2.00 ÷ 5.3741 vs. $3.70 ÷ 11.526 per participating GPU-hour. + metricRow('dollarsPerVideo') + .should('contain', '每条视频 TCO 成本') + .and('contain', '$0.372') + .and('contain', '$0.321'); + metricRow('kjPerVideo').should('contain', '每条视频 GPU 板卡能耗'); + cy.get('[data-testid="video-compare-case-of"]').should('contain', '第 1 /'); + clip('baseline').should('contain', '出片时间 120 s'); + }); +}); diff --git a/packages/app/cypress/component/video-dashboard.cy.tsx b/packages/app/cypress/component/video-dashboard.cy.tsx new file mode 100644 index 000000000..b6f13c7ae --- /dev/null +++ b/packages/app/cypress/component/video-dashboard.cy.tsx @@ -0,0 +1,775 @@ +import { wanServingFixture } from '../../src/components/video-benchmark/serving.fixture'; +import { QUALITY_METRIC_IDS } from '../../src/components/video-benchmark/quality'; +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; +import VideoDashboard from '@/components/video-benchmark/VideoDashboard'; +import { videoHistoryEntry, type VideoHistoryPage } from '@/components/video-benchmark/history'; + +// Retained H100/H200/B200 C1/C2/C4 observations (cypress/fixtures/api/video-history.json). +function mount(pathname = '/video', search = '', history?: VideoHistoryPage) { + cy.intercept( + 'GET', + '/api/video-runs?format=history&page=1', + history + ? { body: history } + : { + fixture: 'api/video-history.json', + }, + ).as('history'); + cy.intercept('GET', '/api/video-runs?page=*', { runs: [], nextPage: null }); + // The Compare panel fetches clips once in view; the component runner publishes none. + cy.intercept('GET', '/api/video-runs?run=*', { statusCode: 204 }); + cy.window().then((win) => + win.history.replaceState(null, '', `${win.location.pathname}${search}`), + ); + cy.mount( + + + , + ); + cy.wait('@history'); +} +const kpi = (hardware: string) => + cy.get(`[data-testid="video-kpi-card"][data-hardware="${hardware}"]`); +const points = () => cy.get('[data-testid="video-hardware-chart"] circle.point'); +const lines = () => cy.get('[data-testid="video-hardware-chart"] path.line-path'); +const tableRows = () => cy.get('[data-testid="video-points-table"] tbody tr'); +const headers = () => cy.get('[data-testid="video-points-table"] thead th'); +// Roster order (HW_REGISTRY sort): the B200 and H100 jobs reserved 8 boards and generated on 4. +const IDLE_NOTE = + 'Per participating GPU: B200 (4 of 8), H100 (4 of 8) reserved more boards than one video uses; the idle boards are not counted.'; +const IDLE_NOTE_ZH = + '按参与计算的 GPU 计:B200(4 / 8 张)、H100(4 / 8 张)预留的板卡多于单条视频所需,空闲板卡未计入。'; +const ALL_COLUMNS = [ + 'Hardware', + 'Deployment', + 'Valid / scheduled', + 'P50 time to video (s)', + 'P90 time to video (s)', + 'Videos per GPU-hour', + 'Videos per $1 TCO (Owning at Large Hyperscaler Volume)', + 'TCO cost per video (Owning at Large Hyperscaler Volume)', + 'API list price per video', + 'GPU-board energy per video (kJ)', + 'Mean board power / enforced limit (%)', + 'Prompt adherence (0–4)', + 'Board power (W)', + 'CI run', + 'Details', +]; + +describe('Video hardware dashboard (retained fixture)', () => { + for (const [width, pathname, toggle] of [ + [1280, '/video', 'Show or hide serving evidence'], + [390, '/zh/video', '展开或收起服务结果记录'], + ] as const) { + it(`retains serving outcomes when quality filters empty the chart at ${width}px`, () => { + cy.viewport(width, 844); + cy.fixture('api/video-history.json').then((page: VideoHistoryPage) => { + // Controlled accounting fixture; no fabricated ratings enter the retained replay. + const source = page.entries + .flatMap((entry) => entry.sources) + .find((s) => s.hardware === 'H200')!; + source.serving = [ + { + cell: 'c1', + concurrency: 1, + status: 'complete', + workloadKey: null, + mode: 'closed_loop', + scheduled: 20, + attempted: 20, + completed: 20, + failed: 0, + timedOut: 0, + notStarted: 0, + valid: 20, + unjudged: 20, + legacyFailedSlots: 0, + unfinished: 0, + deliveryDeadlineSeconds: null, + qualitySloGoodput: null, + provenance: 'request-ledger', + }, + ]; + page.entries[0].sources.push({ + ...source, + id: 'amd-not-started', + hardware: 'MI355X', + observations: [], + serving: [ + { + ...source.serving[0], + status: 'failed', + attempted: 0, + completed: 0, + valid: 0, + unjudged: 0, + notStarted: 20, + legacyFailedSlots: 20, + }, + ], + }); + mount(pathname, '?v_y=quality&v_qmin=4&v_hidden=h200,mi355x', page); + }); + points().should('not.exist'); + cy.get(`[aria-label="${toggle}"]`).click(); + cy.get('[data-testid="video-serving-evidence"]').within(() => { + cy.get('[data-source="amd-not-started"]').within(() => { + cy.get('[data-testid="video-serving-scheduled"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-attempted"]').should('have.text', '0'); + cy.get('[data-testid="video-serving-failed"]').should('have.text', '0'); + cy.get('[data-testid="video-serving-notStarted"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-qualitySloGoodput"]').should('have.text', '—'); + cy.get('a').should('have.attr', 'href').and('include', 'source=amd-not-started'); + }); + cy.contains('[data-testid="video-serving-row"]', 'H200').within(() => { + cy.get('[data-testid="video-serving-valid"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-unjudged"]').should('have.text', '20'); + }); + cy.get('[role="region"]').should(($region) => { + const region = $region[0]; + expect(region.getBoundingClientRect().width).to.be.at.most(width); + expect(region.scrollWidth).to.be.at.least(region.clientWidth); + }); + }); + cy.document().should((doc) => { + expect(doc.documentElement.scrollWidth).to.be.at.most(width); + }); + cy.get(`[aria-label="${toggle}"]`).click(); + cy.get('[data-testid="video-serving-evidence"]').should('not.exist'); + }); + } + for (const width of [1280, 390]) { + it(`keeps six deployment labels inside the plot without overlap at ${width}px`, () => { + cy.viewport(width, 844); + cy.fixture('api/video-history.json').then((page: VideoHistoryPage) => { + // Synthetic near-neighbor frontiers exercise layout only, not measured claims. + for (const entry of page.entries) + for (const source of entry.sources) { + const base = source.observations.find((o) => o.concurrency === 1); + source.observations = + base && (base.hardware.includes('B200') || base.hardware.includes('H200')) + ? [2, 4, 8].map((gpus, index) => { + const b200 = base.hardware.includes('B200'); + const rate = [6.67, 6, 5.29][index] + (b200 ? 0.15 : 0); + return { + ...base, + id: `${base.id}-${gpus}g`, + participating: gpus, + allocated: gpus, + server: { ...base.server, tp: 2, ulysses: gpus / 2, attention: null }, + p90: [290, 150, 85][index] - (b200 ? 5 : 0), + wallSeconds: (20 * 3600) / (gpus * rate), + }; + }) + : []; + } + mount('/video', '?v_y=videosPerGpuHour', page); + }); + points().should('have.length', 6); + lines().should('have.length', 2); + cy.get('[data-testid="video-hardware-chart"] text.video-point-label') + .should('have.length', 6) + .should(($labels) => { + const clip = $labels[0].ownerSVGElement!.querySelector( + '#clip-video-hardware rect', + )!; + const plotWidth = clip.width.baseVal.value; + const plotHeight = clip.height.baseVal.value; + const boxes = [...$labels].map((label) => label.getBBox()); + boxes.forEach((box, index) => { + expect(box.width, `label ${index} width`).to.be.greaterThan(0); + expect(box.x, `label ${index} left`).to.be.at.least(-0.1); + expect(box.y, `label ${index} top`).to.be.at.least(-0.1); + expect(box.x + box.width, `label ${index} right`).to.be.at.most(plotWidth + 0.1); + expect(box.y + box.height, `label ${index} bottom`).to.be.at.most(plotHeight + 0.1); + for (const other of boxes.slice(index + 1)) + expect( + box.x < other.x + other.width && + box.x + box.width > other.x && + box.y < other.y + other.height && + box.y + box.height > other.y, + `label ${index} overlaps another label`, + ).to.equal(false); + }); + }); + }); + } + it('keeps unjudged quality empty before dominance and restores the performance view', () => { + mount('/video', '?v_y=quality&v_quality=audio_quality&v_qmin=4&v_optimal=0'); + points().should('not.exist'); + cy.get('[data-testid="video-quality-summary"]') + .should('contain', 'Audio quality') + .and('contain', 'No calibrated quality results'); + cy.get('[data-testid="video-hardware-chart"]').should('contain', 'Unjudged or uncalibrated'); + cy.contains('button', 'Table').click(); + tableRows().should('not.exist'); + cy.get('[aria-label="Quality threshold (minimum)"]').click(); + cy.contains('[role="option"]', 'Off — descriptive performance').click(); + cy.get('[aria-label="Y-axis metric"]').click(); + cy.contains('[role="option"]', 'Videos per GPU-hour').click(); + tableRows().should('have.length', 3); + cy.location('search').should('not.contain', 'v_qmin'); + cy.contains('button', 'Details').first().click(); + cy.get('[data-testid="video-point-details"]') + .should('contain', '20 / 20 / 20 / 0') + .and('contain', 'SHA256 / commit'); + }); + it('plots only synthetic calibrated quality and filters it at the reader threshold', () => { + // This fabricated judgment tests behavior only; never saved to the measured replay. + cy.fixture('api/video-history.json').then((page: VideoHistoryPage) => { + for (const entry of page.entries) + for (const source of entry.sources) + for (const o of source.observations) { + if (!o.hardware.includes('B200') || o.concurrency !== 1) continue; + o.quality = { + scale: 'ordinal_0_to_4', + contractId: 'synthetic-only', + contractSha256: 'a'.repeat(64), + rubricVersion: 'test-v1', + rubricSha256: 'b'.repeat(64), + metrics: Object.fromEntries( + QUALITY_METRIC_IDS.map((id) => [ + id, + { + value: 3, + status: 'pass', + direction: 'higher', + evaluatorId: 'synthetic-reviewer', + evaluatorVersion: 'test-v1', + evaluatorSha256: 'c'.repeat(64), + samples: o.samples, + total: o.samples, + calibration: { + status: 'calibrated', + cohortId: 'synthetic-only', + threshold: 3, + provenance: 'fixture://rule', + frozenAt: '2026-09-01T00:00:00Z', + }, + }, + ]), + ), + }; + } + mount('/video', '?v_y=quality', page); + points().should('have.length', 1); + lines().should('not.exist'); + cy.get('[data-testid="video-quality-summary"]') + .should('contain', 'synthetic-reviewer@test-v1') + .and('contain', '20/20'); + cy.viewport(390, 844); + cy.contains('button', 'Table').click(); + headers().should(($ths) => { + expect([...$ths].map((el) => el.textContent?.trim())).to.deep.equal([ + 'Hardware', + 'Deployment', + 'P90 time to video (s)', + 'Prompt adherence (0–4)', + 'Details', + ]); + }); + tableRows().should('have.length', 1).find('td').eq(2).should('have.text', '78.3'); + tableRows().find('td').eq(3).should('have.text', '3'); + cy.contains('button', 'Chart').click(); + cy.get('[aria-label="Quality threshold (minimum)"]').click(); + cy.contains('[role="option"]', '≥ 4 / 4').click(); + points().should('not.exist'); + cy.location('search').should('contain', 'v_qmin=4'); + }); + }); + it('plots one C1 point per measured hardware and lists MI355X as unavailable', () => { + mount(); + points().should('have.length', 3); + lines().should('have.length', 0); + cy.get('[data-testid="video-hardware-chart"] path.roofline-path').should('not.exist'); + cy.get('[data-testid="video-chart-caption"]').should( + 'have.text', + 'Measured observations are shown as separate points (4 GPU · TP2 × Ulysses 2). A frontier line requires distinct measured deployments for the same hardware within a comparable cohort.', + ); + cy.get('[data-testid="video-chart-card"]') + .should( + 'contain', + 'MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · Videos per $1 TCO (Owning at Large Hyperscaler Volume) vs. P90 time to video (s)', + ) + .and('contain', 'Cost tier: Owning at Large Hyperscaler Volume'); + cy.get('[data-testid="video-kpi-card"]').should('have.length', 4); + kpi('mi355x').find('[data-testid="video-kpi-unavailable"]').should('contain', 'Not measured'); + // H200: 20 valid ÷ (3013.4 s × 4 participating GPUs) = 5.97 videos/GPU-hr; $1.22 ÷ 5.9732 = $0.204. + kpi('h200') + .should('contain', '4 of 4 GPUs · TP2 × Ulysses 2') + .and('contain', '150.6') + .and('contain', 'P90 151.1') + .and('contain', '5.97') + .and('contain', '$0.204') + .and('contain', '411'); + kpi('b200').should('contain', '11.53').and('contain', '$0.150').and('contain', '301.1'); + // H100 divides by the 4 boards that generated, not the 8 the job reserved: $1.17 ÷ 5.3741. + kpi('h100') + .should('contain', '4 of 8 GPUs · TP2 × Ulysses 2') + .and('contain', '5.37') + .and('contain', '$0.218') + .and('contain', '431.1'); + kpi('h100').should('not.contain', 'Profit').and('not.contain', 'Board power'); + }); + it('selects the model while workload and deployment stay measured facts; the legend offers only Optimal Only', () => { + mount(); + cy.get('[data-testid="video-config-fact"]').should(($facts) => { + expect([...$facts].map((el) => el.textContent)).to.deep.equal([ + // The native H200 run (#39002901777) measures a 4 s plan with four clips per cell, so it stays a hidden workload. + 'Workload1344 × 768 · 8 s · 24 fps · 50 steps (1 other workloads hidden)', + 'Deployment4 GPU · TP2 × Ulysses 2', + ]); + }); + cy.get('[data-testid="video-config-bar"] [role="combobox"]').should(($boxes) => { + expect([...$boxes].map((el) => el.getAttribute('aria-label'))).to.deep.equal([ + 'Model', + 'X-axis metric', + 'Y-axis metric', + 'Quality dimension', + 'Quality threshold (minimum)', + ]); + }); + cy.get('[role="combobox"][aria-label="GPU basis"]').should('not.exist'); + // Same chrome as /inference: title, description and Share in one card above the panels, + // the cost tier picked in the chart caption, Compare and the evidence folded away. + cy.get('[data-testid="video-dashboard"] h1').should('contain', 'VideoGenX · MiniMax-H3'); + cy.get('[data-testid="share-button"]').should('exist'); + cy.get('[data-testid="video-config-bar"]').should('not.contain', 'Cost tier'); + cy.get('[data-testid="video-cost-tier"]').should( + 'contain', + 'Owning at Large Hyperscaler Volume', + ); + cy.get('[data-testid="video-compare"]').should('not.exist'); + cy.get('[data-testid="video-compare-toggle"]').should('have.attr', 'aria-expanded', 'false'); + cy.get('[data-testid="video-evidence-toggle"]').should('have.attr', 'aria-expanded', 'false'); + cy.contains('[data-testid="video-dashboard"] h2', 'Compare'); + cy.contains('[data-testid="video-dashboard"] h2', 'Performance evidence'); + cy.get('[data-testid="video-legend"]').should('contain', 'H100').and('contain', 'MI355X'); + cy.get('[data-testid="video-legend"] [role="switch"]').should('have.length', 1); + cy.get('[data-testid="video-optimal-only"]').should('have.attr', 'aria-checked', 'true'); + cy.get('[data-testid="video-legend"]').should('contain', 'Optimal Only'); + for (const id of ['video-queue', 'video-optimal', 'video-frontier']) + cy.get(`[data-testid="${id}"]`).should('not.exist'); + cy.get('[data-testid="video-idle-note"]').should('have.text', IDLE_NOTE); + }); + it('switches the cost tier, repricing the cards, the badges and the URL', () => { + mount(); + cy.get('[data-testid="video-cost-tier"]').click(); + cy.get('[data-testid="video-cost-tier-r"]').click(); + // $2.90/GPU-hr ÷ 5.9732 videos/GPU-hr = 0.48550 → three decimals. + kpi('h200').should('contain', '$0.485'); + kpi('h100').should('contain', '$0.372'); + kpi('b200').should('contain', '$0.321'); + cy.location('search').should('contain', 'v_tier=r'); + cy.get('[data-testid="video-chart-card"]') + .should('contain', 'Cost tier: Rent - 3 Year Commit') + .and('contain', 'Videos per $1 TCO (Rent - 3 Year Commit)'); + cy.get('[data-testid="video-tco-badge"]') + .first() + .should('contain', 'B200') + .and('contain', '3.70'); + }); + it('never plots queued cells, even when a stale URL asks for them', () => { + // C2/C4 exceed the single replica, so they stay in the evidence panel; the retired + // v_queue/v_opt/v_frontier/v_basis params change neither what is drawn nor what is billed. + mount('/video', '?v_queue=1&v_opt=1&v_frontier=1&v_basis=allocated'); + points().should('have.length', 3); + lines().should('have.length', 0); + cy.get('[data-testid="video-chart-caption"]').should('not.contain', 'queued'); + kpi('h100').should('contain', '$0.218'); + cy.contains('button', 'Table').click(); + tableRows().should('have.length', 3); + cy.get('[data-testid="video-points-table"]').should('not.contain', 'queued'); + // Only C1 is scheduled once per replica: every row shows the full 20 / 20 cohort once. + cy.get('[data-testid="data-table-preset-all"]').click(); + tableRows().each(($row) => expect($row.text()).to.include('20 / 20')); + cy.get('[data-testid="video-idle-note"]').should('have.text', IDLE_NOTE); + }); + it('keeps the selected P90 and GPU-hour metrics in the compact table at phone width', () => { + cy.viewport(390, 844); + mount('/video', '?v_view=table&v_y=videosPerGpuHour'); + headers().should(($ths) => { + expect([...$ths].map((el) => el.textContent?.trim())).to.deep.equal([ + 'Hardware', + 'Deployment', + 'P90 time to video (s)', + 'Videos per GPU-hour', + 'Details', + ]); + }); + cy.contains('[data-testid="video-points-table"] tbody tr', 'B200').should(($row) => { + expect([...$row.find('td')].map((el) => el.textContent?.trim())).to.deep.equal([ + 'B200', + '4 GPU · TP2 × Ulysses 2', + '78.3', + '11.53', + 'Details', + ]); + }); + }); + it('lists the plotted cells with the full column set on demand and no concurrency column', () => { + mount('/video', '?v_view=table'); + cy.get('[data-testid="video-hardware-chart"]').should('not.exist'); + tableRows().should('have.length', 3); + headers().should(($ths) => { + expect([...$ths].map((el) => el.textContent?.trim())).to.deep.equal([ + ...ALL_COLUMNS.slice(0, 2), + ALL_COLUMNS[4], + ALL_COLUMNS[6], + 'Details', + ]); + }); + cy.get('[data-testid="data-table-preset-all"]').click(); + headers().should(($ths) => { + expect([...$ths].map((el) => el.textContent?.trim())).to.deep.equal(ALL_COLUMNS); + }); + cy.contains('[data-testid="video-points-table"] tbody tr', 'B200').should(($row) => { + const cells = [...$row.find('td')].map((el) => el.textContent?.trim()); + expect(cells.slice(0, 13)).to.deep.equal([ + 'B200', + '4 GPU · TP2 × Ulysses 2', + '20 / 20', + '77.9', + '78.3', + '11.53', + '6.66', + '$0.150', + '$0.640', + '301.1', + '96.4', + '—', + '3,856', + ]); + expect(cells[13]).to.equal('#34341996789'); + }); + }); + it('hides a hardware from the legend and shows the table view', () => { + mount(); + cy.contains('[data-testid="video-legend"] li', 'H100').click(); + points().should('have.length', 2); + cy.location('search').should('contain', 'v_hidden=h100'); + cy.contains('button', 'Table').click(); + tableRows().should('have.length', 2); + cy.get('[data-testid="video-points-table"]') + .should('contain', 'H200') + .and('contain', 'B200') + .and('not.contain', 'H100'); + cy.location('search').should('contain', 'v_view=table'); + cy.get('[data-testid="video-history-section"]').should('not.have.attr', 'open'); + }); + it('restores hidden hardware from a shared table URL', () => { + mount('/video', '?v_hidden=h100&v_view=table'); + tableRows().should('have.length', 2); + cy.get('[data-testid="video-points-table"]').should('not.contain', 'H100'); + cy.contains('[data-testid="video-legend"] li', 'H100').click(); + tableRows().should('have.length', 3); + cy.location('search').should('not.contain', 'v_hidden='); + }); + it('restores v_ params from the URL and opens the history section for history deep links', () => { + mount('/video', '?v_y=kjPerVideo&v_x=p50Latency&v_tier=r&history-hardware=H200'); + points().should('have.length', 3); + cy.get('[data-testid="video-chart-card"]') + .should('contain', 'GPU-board energy per video (kJ) vs. P50 time to video (s)') + .and('contain', 'Cost tier: Rent - 3 Year Commit'); + kpi('h200').should('contain', '$0.485'); + cy.get('[data-testid="video-history-section"]').should('have.attr', 'open'); + cy.get('[data-testid="video-history"] h1').should('contain', 'Performance history'); + }); + it('reprices the API list price beside the TCO cost, in the cards, the table and the URL', () => { + mount(); + // 0.08 $/video-s × 8 s clip = $0.640, read beside each hardware's TCO cost per video. + kpi('h200').should('contain', 'API list $0.640').and('contain', '$0.204'); + kpi('h100').should('contain', 'API list $0.640'); + kpi('b200').should('contain', 'API list $0.640'); + cy.get('[data-testid="video-chart-card"]').should( + 'contain', + 'API reference: $0.080/video-s (2026-09-24)', + ); + cy.get('[data-testid="video-api-reference-caption"]') + .should('contain', 'Reference $0.080/video-s') + .and('contain', '$0.080–$0.130') + .and('contain', 'captured 2026-09-24') + .and('contain', 'MiniMax platform'); + cy.get('[data-testid="video-api-price"]').clear().type('0.05'); + // $0.400 per 8 s clip; the TCO cost itself does not move. + kpi('h200').should('contain', 'API list $0.400').and('contain', '$0.204'); + kpi('h200').should('not.contain', '$0.640'); + cy.location('search').should('contain', 'v_api=0.05'); + cy.contains('button', 'Table').click(); + cy.get('[data-testid="data-table-preset-all"]').click(); + cy.get('[data-testid="video-points-table"] thead') + .should('contain', 'API list price per video') + .and('contain', 'TCO cost per video (Owning at Large Hyperscaler Volume)') + .and('not.contain', 'Revenue') + .and('not.contain', 'Profit'); + cy.contains('[data-testid="video-points-table"] tbody tr', 'H200') + .should('contain', '$0.400') + .and('contain', '$0.204'); + cy.get('[data-testid="video-api-price-reset"]').click(); + cy.contains('[data-testid="video-points-table"] tbody tr', 'H200').should('contain', '$0.640'); + kpi('h200').should('contain', 'API list $0.640'); + cy.location('search').should('not.contain', 'v_api'); + }); + it('toggles Optimal Only from the legend and restores it from v_optimal', () => { + mount('/video', '?v_optimal=0'); + cy.get('[data-testid="video-optimal-only"]').should('have.attr', 'aria-checked', 'false'); + // Every fixture deployment is on its hardware's frontier, so the switch changes no point here. + points().should('have.length', 3); + cy.get('[data-testid="video-optimal-only"]').click(); + cy.get('[data-testid="video-optimal-only"]').should('have.attr', 'aria-checked', 'true'); + cy.location('search').should('not.contain', 'v_optimal'); + cy.get('[data-testid="video-optimal-only"]').click(); + cy.location('search').should('contain', 'v_optimal=0'); + points().should('have.length', 3); + }); + it('folds Compare and the evidence by default and opens Compare for compare deep links', () => { + mount(); + cy.get('[data-testid="video-compare"]').should('not.exist'); + cy.get('[data-testid="video-compare-toggle"]').click(); + cy.get('[data-testid="video-compare"]').should('exist'); + cy.get('[data-testid="video-compare-toggle"]').should('have.attr', 'aria-expanded', 'true'); + cy.get('[data-testid="video-evidence-toggle"]').click(); + cy.get('[data-testid="video-evidence-toggle"]').should('have.attr', 'aria-expanded', 'true'); + cy.contains('h2', 'Performance evidence').should('have.length', 1); + mount('/video', '?v_cand=h200'); + cy.get('[data-testid="video-compare"]').should('exist'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H200'); + cy.get('[data-testid="video-evidence-toggle"]').should('have.attr', 'aria-expanded', 'false'); + }); + it('shows placeholders while loading instead of a false "Not measured"', () => { + cy.intercept('GET', '/api/video-runs?format=history&page=1', (req) => { + req.reply({ fixture: 'api/video-history.json', delay: 800 }); + }).as('slowHistory'); + cy.intercept('GET', '/api/video-runs?page=*', { runs: [], nextPage: null }); + cy.intercept('GET', '/api/video-runs?run=*', { statusCode: 204 }); + // The AUT keeps the previous test's v_ params; start from the defaults. + cy.window().then((win) => win.history.replaceState(null, '', win.location.pathname)); + cy.mount( + + + , + ); + cy.get('[data-testid="video-kpi-skeleton"]').should('have.length', 4); + cy.get('[data-testid="video-kpi-unavailable"]').should('not.exist'); + cy.get('[data-testid="video-idle-note"]').should('not.exist'); + cy.wait('@slowHistory'); + cy.get('[data-testid="video-kpi-skeleton"]').should('not.exist'); + kpi('h200').should('contain', '$0.204'); + kpi('mi355x').find('[data-testid="video-kpi-unavailable"]').should('exist'); + cy.get('[data-testid="video-idle-note"]').should('have.text', IDLE_NOTE); + }); + for (const width of [1440, 390]) { + it(`keeps unknown allocation readable in cards and history at ${width}px`, () => { + cy.viewport(width, 900); + cy.fixture('api/video-history.json').then((page: VideoHistoryPage) => { + for (const entry of page.entries) + for (const source of entry.sources) + for (const observation of source.observations) observation.allocated = null; + cy.intercept('GET', '/api/video-runs?format=history&page=1', page).as('allocation'); + }); + cy.intercept('GET', '/api/video-runs?page=*', { runs: [], nextPage: null }); + cy.intercept('GET', '/api/video-runs?run=*', { statusCode: 204 }); + cy.window().then((win) => + win.history.replaceState(null, '', `${win.location.pathname}?history-hardware=H200`), + ); + cy.mount( + + + , + ); + cy.wait('@allocation'); + kpi('h200').should('contain', '4 of — GPUs').and('contain', '5.97'); + cy.get('[data-testid="video-idle-note"]').should('not.exist'); + cy.get('[data-testid="video-history-section"]').should('have.attr', 'open'); + cy.get('[data-testid="video-history-observation"]').should('have.length', 3); + cy.get('[data-testid="video-history"] input').clear().type('71de97b264b0'); + cy.get('[data-testid="video-history-observation"]').should('have.length', 3); + cy.get('[data-testid="video-history-observation"]').first().should('contain', 'n=4'); + cy.get('[data-testid="video-history-power"]').should('contain', 'Measured'); + cy.get('[data-testid="video-history"]').should(($history) => { + expect($history[0].getBoundingClientRect().width).to.be.at.most(width); + }); + cy.screenshot(`video-unknown-allocation-${width}`); + }); + } + it('states each published source’s GPU power verdict in the history list', () => { + // Native srt-slurm sources record power_verified; retained supervisor sources do not. + cy.fixture('api/video-history.json').then((page) => { + page.entries[0].sources[0].powerVerified = true; + page.entries[2].sources[0].powerVerified = false; + cy.intercept('GET', '/api/video-runs?format=history&page=1', page).as('verdicts'); + }); + cy.intercept('GET', '/api/video-runs?page=*', { runs: [], nextPage: null }); + cy.intercept('GET', '/api/video-runs?run=*', { statusCode: 204 }); + cy.window().then((win) => + win.history.replaceState(null, '', `${win.location.pathname}?history-query=`), + ); + cy.mount( + + + , + ); + cy.wait('@verdicts'); + cy.get('[data-testid="video-history-entry"]').should('have.length', 7); + cy.get('[data-testid="video-history-power"]').should(($verdicts) => { + expect([...$verdicts].map((el) => el.textContent?.trim())).to.deep.equal([ + '· GPU power: Measured', + '· GPU power: Not measured', + // The retained native H200 run records its own verdict. + '· GPU power: Measured', + ]); + }); + }); + it('renders Chinese copy on /zh/video', () => { + mount('/zh/video'); + cy.get('[data-testid="video-chart-card"]') + .should('contain', '每 1 美元 TCO 生成视频数(Hyperscaler 自有设备)') + .and('contain', '成本档位: Hyperscaler 自有设备'); + cy.get('[data-testid="video-chart-caption"]').should( + 'have.text', + '实测观测以独立散点展示(4 张 GPU · TP2 × Ulysses 2)。前沿连线需要同一硬件、同一可比组内不同部署的实测结果。', + ); + cy.get('[data-testid="video-idle-note"]').should('have.text', IDLE_NOTE_ZH); + kpi('mi355x').should('contain', '未测得'); + kpi('h200') + .should('contain', '4 / 4 张 GPU') + .and('contain', 'P50 出片时间(s)') + .and('contain', '视频数 / GPU 小时') + .and('contain', 'TCO / 条视频') + .and('contain', 'API 标价 $0.640') + .and('contain', 'kJ / 条视频'); + cy.get('[data-testid="video-config-fact"]').should(($facts) => { + expect([...$facts].map((el) => el.textContent)).to.deep.equal([ + '工作负载1344 × 768 · 8 s · 24 fps · 50 steps(另有 1 个工作负载未显示)', + '部署4 张 GPU · TP2 × Ulysses 2', + ]); + }); + cy.get('[data-testid="video-cost-tier"]').should('contain', 'Hyperscaler 自有设备'); + cy.get('[data-testid="video-legend"]').should('contain', '仅最优'); + cy.contains('[data-testid="video-dashboard"] h2', '对比'); + cy.contains('[data-testid="video-dashboard"] h2', '性能测量证据'); + cy.get('[data-testid="video-config-bar"]') + .should('contain', 'X 轴指标') + .and('contain', 'Y 轴指标') + .and('not.contain', '成本档位') + .and('contain', 'API 参考价($/video-s)') + .and('contain', '采集于 2026-09-24'); + cy.contains('button', '表格').click(); + headers().should(($ths) => { + expect([...$ths].map((el) => el.textContent?.trim())).to.deep.equal([ + '硬件', + '部署', + 'P90 出片时间(s)', + '每 1 美元 TCO 生成视频数(Hyperscaler 自有设备)', + '详情', + ]); + }); + }); +}); + +describe('Wan model evidence isolation', () => { + it('keeps a zero-observation failed source visible without borrowing H3 results', () => { + cy.fixture('api/video-history.json').then((page: VideoHistoryPage) => { + // Synthetic identity and failure for a rendering regression, not a measurement. + const source = structuredClone(page.entries[0].sources[0]); + source.id = 'wan-failure'; + source.model = 'Wan-AI/Wan2.2-T2V-A14B'; + source.observations = []; + source.serving = []; + source.error = 'Generation did not start'; + page.entries[0].sources.push(source); + mount('/video', '?v_model=wan22', page); + }); + cy.get('[data-testid="video-model-empty"]').should('be.visible'); + cy.get('[aria-label="Quality dimension"]').should('not.exist'); + cy.get('[aria-label="Show or hide serving evidence"]').click(); + cy.get('[data-testid="video-serving-row"]') + .should('have.length', 1) + .and('have.attr', 'data-source', 'wan-failure'); + cy.get('[data-testid="video-serving-qualitySloGoodput"]').should('have.text', '—'); + points().should('not.exist'); + }); +}); + +describe('Generic video-only producer (synthetic contract)', () => { + for (const [width, pathname, toggle] of [ + [1280, '/video', 'Show or hide serving evidence'], + [390, '/zh/video', '展开或收起服务结果记录'], + ] as const) { + it(`retains canonical Wan success and preflight outcomes at ${width}px`, () => { + cy.viewport(width, 844); + const producer = wanServingFixture(); + producer.documents.set('manifest.json', producer.manifest); + producer.documents.set('ci.json', producer.ci); + producer.checksums.set('manifest.json', 'a'.repeat(64)); + const entry = () => + videoHistoryEntry( + { + storageVersion: 1, + runId: '123', + artifact: { id: 40, name: 'video-serving-123-1', expired: false, size_in_bytes: 100 }, + sources: [ + { + id: '123', + documents: [...producer.documents], + checksums: [...producer.checksums], + assets: [], + texts: [], + }, + ], + }, + null, + ); + mount(pathname, '?v_model=wan22', { schemaVersion: 1, entries: [entry()], nextPage: null }); + // The default Pareto view retains only C1; all three planned cells remain in serving. + points().should('have.length', 1); + cy.get(`[aria-label="${toggle}"]`).click(); + cy.get('[data-testid="video-serving-row"]').should('have.length', 3); + cy.get('[data-testid="video-serving-row"]') + .first() + .within(() => { + cy.get('[data-testid="video-serving-valid"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-unjudged"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-qualitySloGoodput"]').should('have.text', '—'); + }); + cy.get('[data-testid="video-serving-evidence"] [role="region"]').should(($region) => { + expect($region[0].getBoundingClientRect().width).to.be.at.most(width); + }); + cy.document().its('documentElement.scrollWidth').should('be.lte', width); + // No GPU evidence is fabricated for this preflight failure. + cy.then(() => { + Object.assign(producer.documents.get('serving-smoke.json')!, { + gpu_uuids: [], + cells: [ + { + concurrency: 1, + status: 'failed', + verified: false, + completion: { + scheduled: 20, + attempted: 0, + completed: 0, + valid: 0, + failed: 20, + not_started: 20, + }, + }, + ], + }); + mount(pathname, '?v_model=wan22', { schemaVersion: 1, entries: [entry()], nextPage: null }); + }); + points().should('not.exist'); + cy.get(`[aria-label="${toggle}"]`).click(); + cy.get('[data-testid="video-serving-row"]') + .should('have.length', 1) + .within(() => { + cy.get('[data-testid="video-serving-scheduled"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-attempted"]').should('have.text', '0'); + cy.get('[data-testid="video-serving-notStarted"]').should('have.text', '20'); + cy.get('[data-testid="video-serving-failed"]').should('have.text', '0'); + }); + cy.document().its('documentElement.scrollWidth').should('be.lte', width); + }); + } +}); diff --git a/packages/app/cypress/component/video-evidence.cy.tsx b/packages/app/cypress/component/video-evidence.cy.tsx new file mode 100644 index 000000000..a142e36ba --- /dev/null +++ b/packages/app/cypress/component/video-evidence.cy.tsx @@ -0,0 +1,241 @@ +import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; +import type { VideoHistoryPage } from '@/components/video-benchmark/history'; +import type { VideoPoint } from '@/components/video-benchmark/metrics'; +import { videoPoints } from '@/components/video-benchmark/points'; +import VideoEvidence from '@/components/video-benchmark/VideoEvidence'; + +// Retained H100/H200/B200 C1/C2/C4 observations (cypress/fixtures/api/video-history.json). +// Nothing in the panel is priced, so it takes no metric options: every per-GPU number +// divides by the boards that generated the clip. +function mount(pathname = '/video', select = (points: VideoPoint[]) => points) { + cy.fixture('api/video-history.json').then((page: VideoHistoryPage) => { + cy.mount( + + (key === 'b200' ? 'rgb(1, 2, 3)' : 'rgb(4, 5, 6)')} + /> + , + ); + }); +} +const exhibit = (name: string) => cy.get(`[data-testid="video-evidence-${name}"]`); + +describe('Video performance evidence (retained fixture)', () => { + for (const { locale, width } of [ + { locale: 'en', width: 1280 }, + { locale: 'zh', width: 390 }, + ]) { + it(`keeps the concurrency table within the stated shared layout (${locale}, ${width}px)`, () => { + cy.viewport(width, 844); + mount(locale === 'zh' ? '/zh/video' : '/video', (points) => + points.map((p) => + p.hardwareKey === 'b200' + ? { ...p, participating: 8, server: { tp: 4, ulysses: 2, attention: null } } + : p, + ), + ); + exhibit('plateau').find('tbody tr').should('have.length', 6); + exhibit('plateau').find('tr[data-hardware="b200"]').should('not.exist'); + exhibit('plateau') + .find('tr[data-hardware="h100"][data-concurrency="4"]') + .should('contain', '5.37') + .and('contain', '4.00×'); + exhibit('caveats').should( + 'contain', + locale === 'zh' + ? '每条视频由 4 张 GPU 参与计算(TP2 × Ulysses 2)' + : '4 participating GPUs per video (TP2 × Ulysses 2)', + ); + cy.get('[data-testid="video-evidence"]').should(($section) => { + expect($section[0].scrollWidth).to.be.at.most($section[0].clientWidth); + }); + exhibit('plateau').screenshot(`video-evidence-cohort-${locale}-${width}`); + }); + } + it('does not call low board power compute saturation', () => { + mount('/video', (points) => + points + .filter((p) => p.hardwareKey === 'h200' && p.concurrency === 1) + .map((p) => ({ ...p, avgPowerW: 280 })), + ); + exhibit('power-reading') + .should('contain', '10%') + .and('not.contain', 'near the cap') + .and('not.contain', 'compute-saturated'); + cy.get('[data-testid="video-evidence"]').should('not.contain', 'Compute-bound evidence'); + }); + it('does not call higher throughput and lower latency a queueing plateau', () => { + mount('/video', (points) => { + const base = points.find((p) => p.hardwareKey === 'h200' && p.concurrency === 1)!; + return [ + base, + { + ...base, + id: 'synthetic-c2', + concurrency: 2, + wallSeconds: base.wallSeconds! / 2, + p50: base.p50! / 2, + }, + ]; + }); + exhibit('plateau-reading') + .should('contain', '100.0%') + .and('contain', '0.50× at C2') + .and('not.contain', 'rises') + .and('not.contain', 'queues requests'); + exhibit('plateau').should('not.contain', 'Client concurrency plateau'); + }); + it('reads board power against the recorded enforced limit per measured hardware', () => { + mount(); + exhibit('power').find('[data-testid="video-evidence-power-row"]').should('have.length', 3); + exhibit('power') + .find('[data-hardware="h100"]') + .should('contain', '2,575 W / 2,800 W') + .and('contain', '91.9%'); + exhibit('power').find('[data-hardware="h200"]').should('contain', '97.4%'); + exhibit('power') + .find('[data-hardware="b200"]') + .should('contain', '3,856 W / 4,000 W') + .and('contain', '96.4%'); + exhibit('power').find('[role="meter"]').should('have.length', 3); + exhibit('power') + .find('[data-hardware="b200"] [role="meter"]') + .should('have.attr', 'aria-valuenow', '96.4') + .find('div') + .should('have.css', 'background-color', 'rgb(1, 2, 3)'); + exhibit('power-reading').should( + 'contain', + 'All 3 measured GPU types ran at 92–97% of their enforced power limit', + ); + }); + it('bounds the power meter to its declared range when mean power exceeds the recorded limit', () => { + // 2,856 W against the recorded 2,800 W limit: the label keeps the measured 102.0%, + // the meter reports and fills its 0–100 range. + mount('/video', (points) => + points + .filter((p) => p.hardwareKey === 'h200' && p.concurrency === 1) + .map((p) => ({ ...p, avgPowerW: 2856 })), + ); + exhibit('power') + .find('[data-hardware="h200"]') + .should('contain', '2,856 W / 2,800 W') + .and('contain', '102.0%'); + exhibit('power') + .find('[data-hardware="h200"] [role="meter"]') + .should('have.attr', 'aria-valuemax', '100') + .and('have.attr', 'aria-valuenow', '100') + .find('div') + .invoke('attr', 'style') + .should('match', /width:\s*100%/u); + exhibit('power-reading').should('contain', 'H200 ran at 102% of its enforced power limit'); + }); + it('tabulates concurrency measurements with ratios against each hardware C1 cell', () => { + mount(); + exhibit('plateau').find('tbody tr').should('have.length', 9); + exhibit('plateau').find('tbody th[scope="rowgroup"]').should('have.length', 3); + // Throughput divides by the 4 participating boards, never the 8 the B200/H100 jobs reserved. + exhibit('plateau') + .find('tr[data-hardware="b200"][data-concurrency="4"]') + .should('contain', '11.69') + .and('contain', '307.3') + .and('contain', '1.01×') + .and('contain', '3.94×'); + exhibit('plateau') + .find('tr[data-hardware="h100"][data-concurrency="1"]') + .should('contain', '5.37') + .and('contain', '167.3') + .and('contain', '1.00×'); + exhibit('plateau') + .find('tr[data-hardware="h100"][data-concurrency="4"]') + .should('contain', '5.37') + .and('contain', '669.2') + .and('contain', '1.00×') + .and('contain', '4.00×'); + exhibit('plateau') + .find('tr[data-hardware="h200"][data-concurrency="1"]') + .should('contain', '5.97') + .and('contain', '150.6'); + exhibit('plateau-reading') + .should('contain', 'Across 6 cells above C1') + .and('contain', 'largest deviation in throughput per GPU-hour from C1 is 1.4%') + .and('contain', '1.97–2.00× at C2 and 3.94–4.00× at C4') + .and('contain', 'These ratios alone do not establish queueing or batching'); + }); + it('compares observed speedups with spec-sheet ratios without attributing a bottleneck', () => { + mount(); + exhibit('scaling').find('[data-testid="video-evidence-scaling-row"]').should('have.length', 2); + exhibit('scaling') + .find('[data-pair="h100-h200"]') + .should('contain', 'H100') + .and('contain', 'H200') + .and('contain', '1.11×') + .and('contain', '1.43×') + .and('contain', '1.00×'); + exhibit('scaling') + .find('[data-pair="h200-b200"]') + .should('contain', '1.93×') + .and('contain', '1.67×') + .and('contain', '2.28×') + .and('contain', '2.27×'); + exhibit('scaling-reading') + .should('contain', 'H100 → H200: 1.11× observed vs. 1.43× bandwidth and 1.00× FLOPS') + .and( + 'contain', + 'H200 → B200: 1.93× observed vs. 1.67× bandwidth and 2.28× (BF16) / 2.27× (FP8) FLOPS', + ) + .and('contain', 'Spec-sheet ratios provide context, not bottleneck attribution'); + exhibit('caveats') + .should('contain', 'Client concurrency is not observed batch size or replica count') + .and('contain', '4 participating GPUs per video (TP2 × Ulysses 2)') + .and('contain', 'the allocated GPU count may differ') + .and('contain', 'not node, rack or facility power') + .and('contain', 'n = 20 clips per cell') + .and('contain', 'Matched workload: 1344 × 768 · 8 s · 24 fps · 50 steps'); + }); + it('renders Chinese copy on /zh/video at mobile width', () => { + cy.viewport(390, 844); + mount('/zh/video'); + cy.get('[data-testid="video-evidence"]') + .should('contain', '性能测量证据') + .and('contain', '板卡功率 vs. 生效功率上限') + .and('contain', '客户端并发测量') + .and('contain', '实测加速比 vs. 规格表比值'); + exhibit('power-reading').should( + 'contain', + '全部 3 种实测硬件生成期间均运行在生效功率上限的 92%–97%', + ); + exhibit('plateau-reading') + .should('contain', '最大偏差为 1.4%') + .and('contain', 'C2 时 1.97–2.00×、C4 时 3.94–4.00×'); + exhibit('scaling-reading').should('contain', '规格表比值仅供参考,不能用于判定瓶颈'); + cy.get('[data-testid="video-evidence"]') + .should('not.contain', '算力饱和') + .and('not.contain', '出片时间升至') + .should(($section) => { + expect($section[0].scrollWidth).to.be.at.most($section[0].clientWidth); + }); + exhibit('caveats') + .should('contain', '每条视频由 4 张 GPU 参与计算(TP2 × Ulysses 2)') + .and('contain', '每个 cell n = 20 条视频'); + cy.screenshot('video-evidence-zh-mobile', { capture: 'viewport' }); + }); + it('hides the plateau and scaling exhibits when only one hardware at C1 is measured', () => { + mount('/video', (points) => + points.filter((p) => p.hardwareKey === 'h200' && p.concurrency === 1), + ); + exhibit('power').find('[data-testid="video-evidence-power-row"]').should('have.length', 1); + exhibit('power-reading').should('contain', 'H200 ran at 97% of its enforced power limit'); + exhibit('plateau').should('not.exist'); + exhibit('scaling').should('not.exist'); + exhibit('caveats').should('contain', '4 participating GPUs per video (TP2 × Ulysses 2)'); + }); + it('renders nothing when no cell is measured', () => { + cy.mount( + + 'rgb(0, 0, 0)'} /> + , + ); + cy.get('[data-testid="video-evidence"]').should('not.exist'); + }); +}); diff --git a/packages/app/cypress/component/video-fidelity.cy.tsx b/packages/app/cypress/component/video-fidelity.cy.tsx deleted file mode 100644 index e13eab67a..000000000 --- a/packages/app/cypress/component/video-fidelity.cy.tsx +++ /dev/null @@ -1,133 +0,0 @@ -import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; -import FidelityResults from '@/components/video-benchmark/FidelityResults'; -import { fidelityFixture } from '@/components/video-benchmark/fidelity.fixture'; -import { at } from '@/components/video-benchmark/bundle'; - -// Synthetic non-playable bytes validate controls, URL binding, and accounting only. -// Actual decoded video/audio playback is verified separately using retained CI output. -const mount = (props: React.ComponentProps, locale = '/video') => - cy.mount( - - - , - ); - -describe('Native paired fidelity UI (synthetic artifacts)', () => { - it('distinguishes valid media from 18 threshold failures and changes both selected videos', () => { - cy.then(() => fidelityFixture({ pairs: 20, failedPairs: 18 })).then((fixture) => { - mount({ published: fixture.published, runId: fixture.runId }); - cy.contains('p', 'Matched valid pairs').next().should('have.text', '20'); - cy.contains('p', 'Pairs outside fidelity thresholds').next().should('have.text', '18'); - cy.contains('dt', 'Threshold calibration').next().should('have.text', 'Uncalibrated'); - cy.contains('dt', 'Release qualified').next().should('have.text', 'No'); - cy.contains('dt', 'Video/audio technical validity').next().should('have.text', 'Yes'); - for (const role of ['Baseline', 'Candidate']) - cy.get(`video[aria-label="${role}"]`) - .should('have.prop', 'controls', true) - .and('have.prop', 'muted', false) - .and('have.attr', 'preload', 'metadata'); - const initial = ['baseline', 'candidate'].map( - (role) => `report/${at(fixture.portable, 'slots', 0, role, 'artifact_path')}`, - ); - const next = ['baseline', 'candidate'].map( - (role) => `report/${at(fixture.portable, 'slots', 1, role, 'artifact_path')}`, - ); - for (const [index, role] of ['Baseline', 'Candidate'].entries()) - cy.get(`video[aria-label="${role}"]`).should( - 'have.attr', - 'src', - fixture.published.assets.find(([path]) => path === initial[index])![1].url, - ); - cy.get('[aria-label="Paired clip"]').click(); - cy.get('[role="option"]').should('have.length', 20); - cy.contains('[role="option"]', 'synthetic-case-2').click(); - for (const [index, role] of ['Baseline', 'Candidate'].entries()) - cy.get(`video[aria-label="${role}"]`).should( - 'have.attr', - 'src', - fixture.published.assets.find(([path]) => path === next[index])![1].url, - ); - cy.contains('Synthetic test prompt 2').should('be.visible'); - cy.contains('a', 'Raw comparison JSON').should( - 'have.attr', - 'href', - fixture.published.assets.find(([path]) => path === 'comparison.json')![1].downloadUrl, - ); - cy.contains('a', 'CI #101').should( - 'have.attr', - 'href', - 'https://github.com/SemiAnalysisAI/InferenceX/actions/runs/101', - ); - }); - }); - - it('rewrites report media/download links and removes scripts and external resources', () => { - cy.then(() => fidelityFixture()).then((fixture) => { - mount({ published: fixture.published, runId: fixture.runId }); - cy.get('iframe').should('not.exist'); - cy.contains('summary', 'Original report').click(); - cy.get('iframe') - .should('have.attr', 'sandbox', 'allow-same-origin allow-downloads') - .should(($iframe) => { - const doc = $iframe[0].contentDocument!; - const path = `report/${at(fixture.portable, 'slots', 0, 'baseline', 'artifact_path')}`; - expect(doc.querySelector('video')?.getAttribute('src')).to.equal( - fixture.published.assets.find(([key]) => key === path)![1].url, - ); - expect(doc.querySelector('a')?.getAttribute('href')).to.equal( - fixture.published.assets.find(([key]) => key === 'report/index.comparison.json')![1] - .url, - ); - expect(doc.querySelectorAll('script')).to.have.length(0); - expect(doc.querySelector('img')?.hasAttribute('src')).to.equal(false); - expect(doc.body.dataset.scriptRan).to.equal(undefined); - }); - cy.contains('summary', 'Original report').click(); - cy.get('iframe').should('not.exist'); - }); - }); - - it('loads a raw sealed bundle and revokes owned object URLs when unmounted', () => { - cy.window().then((win) => { - cy.spy(win.URL, 'revokeObjectURL').as('revoke'); - }); - cy.then(() => fidelityFixture()).then((fixture) => { - mount({ reader: fixture.read, runId: fixture.runId }); - cy.get('[data-testid="fidelity-results"]').should('be.visible'); - cy.get('video[aria-label="Baseline"]') - .invoke('attr', 'src') - .then((url) => { - expect(url).to.match(/^blob:/u); - cy.mount(

Unmounted synthetic reader

); - cy.get('@revoke').should('have.been.calledWith', url); - }); - }); - }); - - it('clears old evidence and reports an invalid or empty comparison', () => { - cy.then(() => fidelityFixture()).then((fixture) => { - mount({ published: fixture.published, runId: fixture.runId }); - cy.get('[data-testid="fidelity-results"]').should('be.visible'); - const invalid = structuredClone(fixture.published); - for (const [path, value] of invalid.documents) - if (['comparison.json', 'report/index.comparison.json'].includes(path)) - Object.assign(value!, { slots: [] }); - mount({ published: invalid, runId: fixture.runId }); - cy.contains('[role="alert"]', 'Could not load paired evidence').should('be.visible'); - cy.get('[data-testid="fidelity-results"]').should('not.exist'); - cy.get('video').should('not.exist'); - }); - }); - - it('renders Chinese status and original-audio controls without relabeling failures as invalid media', () => { - cy.then(() => fidelityFixture()).then((fixture) => { - mount({ published: fixture.published, runId: fixture.runId }, '/zh/video'); - cy.contains('成对视频保真度').should('be.visible'); - cy.contains('p', '超出保真度阈值的视频对').next().should('have.text', '1'); - cy.contains('dt', '视频与音频技术有效性').next().should('have.text', '是'); - cy.contains('dt', '阈值校准状态').next().should('have.text', '未校准'); - cy.get('video[aria-label="基线"]').should('have.prop', 'muted', false); - cy.get('video[aria-label="候选"]').should('have.prop', 'controls', true); - }); - }); -}); diff --git a/packages/app/cypress/component/video-serving-results.cy.tsx b/packages/app/cypress/component/video-serving-results.cy.tsx deleted file mode 100644 index f62b313d8..000000000 --- a/packages/app/cypress/component/video-serving-results.cy.tsx +++ /dev/null @@ -1,399 +0,0 @@ -import { useState } from 'react'; -import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; -import ServingResults from '@/components/video-benchmark/ServingResults'; -import type { ServingCell } from '@/components/video-benchmark/serving'; -import { at, entries, rows, type Bundle } from '@/components/video-benchmark/bundle'; - -// Synthetic fixtures exercise the serving contract; they are never benchmark results. -const cells: ServingCell[] = [1, 2, 4].map((concurrency) => { - const id = `c${concurrency}`; - const request = (phase: string, slot: string) => ({ - phase, - slot_id: slot, - artifact_path: `artifacts/${slot}.mp4`, - prompt: 'Synthetic test prompt only', - seed: 11, - submit_to_media_seconds: 120 * concurrency, - media_validation_seconds: 0.6, - media: { - valid: true, - video: { duration_seconds: 4.4583, width: 1344, height: 768, frame_count: 107, fps: 24 }, - audio: { sample_rate_hz: 32000, channels: 2 }, - checks: [{ name: 'media.decode', status: 'passed' }], - }, - }); - return { - id, - concurrency, - runPath: `gpu/${id}/baseline/run.json`, - jobPath: `gpu/${id}/gpu-job.json`, - powerPath: `gpu/${id}/power.json`, - specPath: `gpu/${id}/spec.json`, - cell: { - status: 'complete', - verified: true, - completion: { - valid: 4, - scheduled: 4, - completed: 4, - failed: 0, - not_started: 0, - unfinished: 0, - }, - metrics: { - client_ready_p50_seconds: 120 * concurrency, - valid_clips_per_second: 1 / 120, - measurement: { wall_seconds: 480 }, - serving: { - client_ready_latency_seconds: { sample_count: 4, p90: null }, - valid_video_seconds_per_second: 4.4583 / 120, - peak_client_in_flight: concurrency, - }, - }, - }, - run: { - records: [request('warmup', 'warmup-001'), request('measurement', 'measurement-r001-c001')], - plan: { generation: { duration_seconds: 4, frame_count: 107, num_inference_steps: 50 } }, - configuration: { - model_id: 'Synthetic model', - model_revision: 'synthetic-model-revision', - runtime_revision: 'synthetic-runtime-revision', - }, - }, - job: { - roles: { - baseline: { - power_configuration_before: { - status: 'recorded', - observed_at: '2026-09-09T01:00:00Z', - gpus: ['GPU-test', 'GPU-test-2'].map((uuid) => ({ - uuid, - configured_limit_w: 700, - enforced_limit_w: 700, - })), - }, - power_configuration_after: { - status: 'recorded', - observed_at: '2026-09-09T02:00:00Z', - gpus: ['GPU-test', 'GPU-test-2'].map((uuid) => ({ - uuid, - configured_limit_w: 700, - enforced_limit_w: 700, - })), - }, - telemetry_summary: { - gpu_identity: [{ name: 'Synthetic GPU', uuid: 'GPU-test' }], - measurement_observed_memory_peak_mib_by_gpu: { 'GPU-test': 100000 }, - }, - }, - }, - }, - spec: { gpu_uuids: ['GPU-test', 'GPU-test-2'], server: { ulysses_degree: 2, tp_size: 1 } }, - power: { - semantics: { power_unit: 'W' }, - phases: { - measurement: { - valid: true, - duration_seconds: 479.9, - aggregate: { - avg_power_w: 1374, - observed_peak_power_w: 1394, - energy_j: 656721, - joules_per_valid_clip: 164180, - }, - per_gpu: { - 'GPU-test': { avg_power_w: 687, observed_peak_power_w: 697 }, - 'GPU-test-2': { avg_power_w: 687, observed_peak_power_w: 697 }, - }, - }, - startup: { - valid: false, - invalid_reasons: ['Synthetic missing telemetry'], - aggregate: { avg_power_w: 999999 }, - }, - }, - }, - }; -}); -const bundle: Bundle = { - manifest: { - run_id: '123', - git_commit: 'synthetic-backend-sha', - resources: { requested: { gpus: 8 } }, - }, - ci: { slurm_job: { AllocTRES: 'cpu=32,mem=512G,gres/gpu=8' } }, - result: null, - job: null, - report: null, - comparison: null, - documents: new Map(), - files: new Map(), - checksums: new Map(), - manifestSha256: 'synthetic-manifest-sha', -}; -const urls = new Map( - cells.flatMap((cell) => - ['measurement-r001-c001', 'warmup-001'].map((slot): [string, string] => [ - `gpu/${cell.id}/baseline/artifacts/${slot}.mp4`, - `https://media.example.test/${cell.id}/${slot}.mp4`, - ]), - ), -); - -function mount( - props: Partial> = {}, - locale = '/video', -) { - cy.mount( - - - , - ); -} - -function LinkedSelection() { - const [cell, setCell] = useState('c1'); - return ( - - - - - ); -} - -describe('H3 serving results (synthetic fixtures)', () => { - it('follows chart-driven cell changes while the result remains mounted', () => { - cy.mount(); - cy.contains('button', 'C1').should('have.attr', 'aria-pressed', 'true'); - cy.get('[aria-label="Clip / request"]').click(); - cy.contains('[role="option"]', 'Warmup').click(); - cy.get('video').should('have.attr', 'src', 'https://media.example.test/c1/warmup-001.mp4'); - cy.contains('button', 'Open C4 from chart').click(); - cy.contains('button', /^C4$/u).should('have.attr', 'aria-pressed', 'true'); - cy.get('video').should( - 'have.attr', - 'src', - 'https://media.example.test/c4/measurement-r001-c001.mp4', - ); - cy.contains('button', 'C2').click(); - cy.contains('button', 'C2').should('have.attr', 'aria-pressed', 'true'); - }); - it('renders a parent-created blob in the report while keeping scripts disabled', () => { - cy.window().then((win) => { - const blobUrl = win.URL.createObjectURL( - new Blob( - [ - '', - ], - { type: 'image/svg+xml' }, - ), - ); - mount({ - html: `Synthetic blob imageDownload synthetic blob`, - }); - cy.get('iframe').should('not.exist'); - cy.contains('summary', 'Original report').click(); - cy.get('iframe') - .should('have.attr', 'sandbox', 'allow-same-origin allow-downloads') - .should(($frame) => { - const doc = $frame[0].contentDocument; - expect(doc?.querySelector('img')?.naturalWidth).to.equal(2); - expect(doc?.querySelector('a')?.getAttribute('href')).to.equal(blobUrl); - expect(doc?.body.dataset.scriptRan).to.equal(undefined); - }) - .then(() => win.URL.revokeObjectURL(blobUrl)); - cy.contains('summary', 'Original report').click(); - cy.get('iframe').should('not.exist'); - }); - }); - - it('shows each concurrency without manufacturing a candidate and separates participating and allocated GPU counts', () => { - const change = cy.stub().as('change'); - mount({ onCellChange: change }); - cy.get('[data-testid="serving-matrix"] tbody tr').should('have.length', 3); - cy.get('[data-testid="serving-matrix"]').within(() => { - cy.contains('th', 'Energy / valid clip (kJ/clip)').should('be.visible'); - cy.contains('td', '164.18').should('be.visible'); - cy.contains('th', 'Mean / enforced limit (%)').should('be.visible'); - cy.contains('td', '98.14').should('be.visible'); - }); - cy.get('[data-testid="serving-selected-metrics"]').within(() => { - cy.contains('120 s').should('be.visible'); - cy.contains('p', '15').should('be.visible'); - cy.contains('dt', 'P90 delivery latency').next().should('have.text', 'Unavailable'); - }); - cy.get('[aria-label="GPU normalization"]').click(); - cy.contains('[role="option"]', 'All allocated GPUs').click(); - cy.get('[data-testid="serving-selected-metrics"]').contains('p', '3.75').should('be.visible'); - cy.get('video') - .should('have.attr', 'src', 'https://media.example.test/c1/measurement-r001-c001.mp4') - .and('have.prop', 'muted', false); - cy.contains('button', /^C4$/u).click(); - cy.get('@change').should('have.been.calledWith', 'c4'); - cy.get('[data-testid="serving-selected-metrics"]').should('contain', '480 s'); - cy.get('video').should( - 'have.attr', - 'src', - 'https://media.example.test/c4/measurement-r001-c001.mp4', - ); - cy.get('[aria-label="Clip / request"]').click(); - cy.contains('[role="option"]', 'Warmup').click(); - cy.get('video').should('have.attr', 'src', 'https://media.example.test/c4/warmup-001.mp4'); - cy.get('[data-testid="serving-selected-metrics"]').should('contain', '4 / 4'); - cy.contains('a', 'Download original MP4').should( - 'have.attr', - 'href', - 'https://media.example.test/c4/warmup-001.mp4', - ); - cy.contains('Baseline').should('not.exist'); - cy.contains('Candidate').should('not.exist'); - }); - - it('shows server stages separately from delivery and retains partial coverage', () => { - const distribution = { - values: [1, 2, 3, 4], - sample_count: 4, - valid_clip_count: 4, - missing_count: 0, - p50: 2.5, - p90: null, - p95: null, - }; - const original = cells[0]; - const record = rows(at(original.run, 'records'))[1]; - const timed = { - ...original, - run: { - ...Object.fromEntries(entries(original.run)), - summary: { valid: 4 }, - serving: { - queue_delay_seconds: distribution, - server_ready_latency_seconds: { - ...distribution, - values: [120, 121], - sample_count: 2, - missing_count: 2, - p50: 999999, - }, - observed_batch_sizes: [1, 1, 1, 1], - observed_replica_ids: [0], - }, - records: [ - { - ...Object.fromEntries(entries(record)), - job_id: 'synthetic-request', - server_timings: { - schema_version: '1.0.0', - status: 'complete', - request_id: 'synthetic-request', - instance_id: 'synthetic-instance', - clock_id: 'synthetic-clock', - clock: 'time.monotonic_ns; same Linux boot and time namespace; nanoseconds', - observed_batch_size: 1, - replica_id: 0, - server_ready_latency_seconds: 119, - prequeue_seconds: 1, - queue_delay_seconds: 0, - execution_seconds: 117, - postprocess_seconds: 1, - }, - }, - ], - }, - }; - mount({ cells: [timed] }); - cy.get('[data-testid="serving-server-timing"]').within(() => { - cy.contains('tr', 'Queue delay').should('contain', '2.5').and('contain', '4 / 4 / 0'); - cy.contains('tr', 'Received → media ready') - .should('contain', 'Unavailable') - .and('contain', '2 / 4 / 2') - .and('not.contain', '999,999'); - cy.contains('dt', 'Observed batch sizes').next().should('have.text', '1'); - cy.contains('dt', 'Observed replica IDs').next().should('have.text', '0'); - }); - cy.get('[data-testid="request-server-timing"]').within(() => { - cy.get('summary').click(); - cy.contains('dt', 'Queue delay').next().should('have.text', '0 s'); - cy.contains('dt', 'Received → media ready').next().should('have.text', '119 s'); - cy.contains('time.monotonic_ns').should('be.visible'); - }); - cy.contains('dt', 'This request: submission → downloaded media') - .next() - .should('have.text', '120 s'); - mount({ cells: [timed] }, '/zh/video'); - cy.get('[data-testid="serving-server-timing"]').within(() => { - cy.contains('tr', '排队时长').should('contain', '2.5'); - cy.contains('tr', '接收请求 → 媒体就绪') - .should('contain', '无数据') - .and('contain', '2 / 4 / 2'); - }); - }); - - it('distinguishes requested and decoded duration and withholds invalid power', () => { - mount(); - cy.contains('dt', 'Requested duration (s)').next().should('have.text', '4'); - cy.contains('dt', 'Decoded video duration (s)').next().should('have.text', '4.4583'); - cy.get('[data-testid="serving-power"]').within(() => { - cy.contains('dt', 'Energy / valid clip (kJ/clip)').next().should('have.text', '164.18'); - cy.get('[aria-label="Power window"]').click(); - }); - cy.contains('[role="option"]', 'Startup').click(); - cy.get('[data-testid="serving-power"]').within(() => { - cy.contains('dt', 'Aggregate mean (W)').next().should('have.text', 'Unavailable'); - cy.contains('dt', 'Mean / enforced limit (%)').next().should('have.text', 'Unavailable'); - cy.contains('Synthetic missing telemetry').should('be.visible'); - cy.contains('999,999').should('not.exist'); - }); - }); - - it('withholds headline metrics and power for an unverified configuration', () => { - mount({ - cells: [ - { - ...cells[0], - cell: { - verified: false, - status: 'failed', - metrics: { client_ready_p50_seconds: 999999, valid_clips_per_second: 999999 }, - }, - }, - ], - }); - cy.get('[data-testid="serving-selected-metrics"]').should('not.contain', '999,999'); - cy.get('[data-testid="serving-matrix"]').should('not.contain', '999,999'); - cy.contains('dt', 'Measurement verified').next().should('have.text', 'No'); - cy.get('[data-testid="serving-power"]').within(() => { - cy.contains('dt', 'Aggregate mean (W)').next().should('have.text', 'Unavailable'); - cy.contains('dt', 'Mean / enforced limit (%)').next().should('have.text', 'Unavailable'); - }); - }); - - it('supports Chinese copy and missing request, media, and allocation data', () => { - mount({ urls: new Map(), bundle: { ...bundle, ci: null } }, '/zh/video'); - cy.contains('并发服务测试结果').should('be.visible'); - cy.contains('此请求暂无可用媒体。').should('be.visible'); - cy.get('video').should('not.exist'); - cy.get('[data-testid="serving-server-timing"]').should('contain', '无数据'); - cy.contains('p', '有效视频 / 参与计算 GPU 小时').next().should('have.text', '15'); - cy.get('[aria-label="GPU 归一化口径"]').click(); - cy.contains('[role="option"]', '所有已分配 GPU').click(); - cy.contains('p', '有效视频 / 已分配 GPU 小时').next().should('have.text', '无数据'); - mount({ cells: [{ ...cells[0], run: null, job: null, spec: null, power: null }] }); - cy.contains('No request records are available for this configuration.').should('be.visible'); - }); -}); diff --git a/packages/app/cypress/component/video-tradeoff.cy.tsx b/packages/app/cypress/component/video-tradeoff.cy.tsx deleted file mode 100644 index 1025b50d9..000000000 --- a/packages/app/cypress/component/video-tradeoff.cy.tsx +++ /dev/null @@ -1,290 +0,0 @@ -import { PathnameContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime'; -import VideoTradeoff from '@/components/video-benchmark/VideoTradeoff'; -import type { TradeoffRun } from '@/components/video-benchmark/tradeoff'; -import { at, entries, rows, type Json } from '@/components/video-benchmark/bundle'; -import { servingFixture } from '@/components/video-benchmark/serving.fixture'; - -const role = { - metrics: { - status: 'valid', - measurement: { wall_seconds: 3600, concurrency: 2 }, - completion: { valid: 10, completed: 10, scheduled: 11, failed: 1 }, - }, - records: Array.from({ length: 10 }, (_, i) => ({ - phase: 'measurement', - status: 'succeeded', - submit_to_media_seconds: i + 1, - media: { valid: true, video: { duration_seconds: 8 } }, - })), -}; -const run: TradeoffRun = { - exportRun: '20', - artifact: '30', - bundle: { - manifest: { run_id: '10' }, - manifestSha256: 'synthetic-only', - result: { - schema_version: '1.0.0', - bundle_type: 'h3_benchmark_result', - workload: { - plan: { - model_id: 'synthetic', - model_revision: 'synthetic-revision', - generation: { - width: 1280, - height: 720, - fps: 24, - duration_seconds: 8, - num_inference_steps: 50, - }, - cases: [{ prompt: 'Synthetic test only', seed: 1 }], - }, - }, - hardware: { - reserved_gpu_count: 8, - selected_gpu_count: 4, - devices: [{ name: 'Synthetic GPU' }], - }, - roles: { baseline: role, candidate: role }, - }, - }, -}; - -describe('Video tradeoff chart (synthetic fixtures)', () => { - it('plots participating GPU efficiency by default and requires assumptions for the cost axis', () => { - const open = cy.stub().as('open'); - cy.mount( - - - , - ); - cy.get('[role="combobox"][aria-label="Efficiency axis · higher is better"]').should( - 'contain', - 'Valid clips / participating GPU-hour', - ); - cy.get('[data-testid="video-tradeoff-chart"] circle.point').should('have.length', 2); - cy.contains('td', '2.5').should('be.visible'); - cy.get('[role="combobox"][aria-label="Efficiency axis · higher is better"]').click(); - cy.contains('[role="option"]', 'Valid clips / allocated GPU-hour').click(); - cy.contains('td', '1.25').should('be.visible'); - cy.contains('dd', '8 / 4').should('be.visible'); - cy.get('[role="combobox"][aria-label="Efficiency axis · higher is better"]').click(); - cy.contains('[role="option"]', 'Valid clips / USD').click(); - cy.contains('No points qualify for these axes yet.').should('be.visible'); - cy.get('input[aria-label="Whole deployment cost (USD/hour)"]').type('2'); - cy.get('circle.point').should('not.exist'); - cy.get('input[aria-label="Cost source and included items"]').type( - 'Synthetic complete-deployment cost', - ); - cy.get('input[aria-label="Cost as of"]').type('2026-09-09'); - cy.get('circle.point').should('have.length', 1); - cy.contains('td', '5').should('be.visible'); - cy.contains('button', 'Open videos and full result').click(); - cy.get('@open').should('have.been.calledOnce'); - }); - it('still withholds P90 for single-request results until median diagnostics are selected', () => { - const single = structuredClone(run); - const result = single.bundle.result as { roles: Record }; - for (const point of Object.values(result.roles)) { - point.records = point.records.slice(0, 1); - point.metrics.completion = { valid: 1, completed: 1, scheduled: 1, failed: 0 }; - } - cy.mount( - - undefined} /> - , - ); - cy.contains('No points qualify for these axes yet.').should('be.visible'); - cy.get('circle.point').should('not.exist'); - cy.contains('button', 'Inspect available serial results').click(); - cy.get('[role="combobox"][aria-label="Latency axis · lower is better"]').should( - 'contain', - 'Median client-ready latency (s)', - ); - cy.get('circle.point').should('have.length', 2); - }); - it('shows an empty state without inventing fixture results', () => { - cy.mount( - - undefined} /> - , - ); - cy.contains('Open a CI result').should('be.visible'); - cy.get('circle.point').should('not.exist'); - }); - it('distinguishes groups with the same short label and exposes their full workload', () => { - const other = structuredClone(run); - other.bundle.manifestSha256 = 'different-synthetic-workload'; - const result = other.bundle.result as { - workload: { plan: { generation: Record } }; - }; - result.workload.plan.generation.guidance_scale = 4; - cy.mount( - - undefined} /> - , - ); - cy.get('[role="combobox"][aria-label="Matched workload"]').click(); - cy.get('[role="option"]').should('have.length', 2); - cy.contains('[role="option"]', '#1').should('be.visible'); - cy.contains('[role="option"]', '#2').click(); - cy.contains('summary', 'Matched workload').click(); - cy.get('[data-testid="tradeoff-detail"] pre').first().should('contain', '"guidance_scale": 4'); - }); -}); - -const matrix = (): TradeoffRun => ({ - exportRun: '123', - artifact: '456', - bundle: { ...servingFixture(), result: null, manifestSha256: 'synthetic-serving' }, -}); - -describe('Video serving matrix chart (synthetic contract fixture)', () => { - it('shows server coverage in point details without changing the client latency axis', () => { - const timed = matrix(); - const documents = timed.bundle.documents!; - const runPath = 'gpu/c1/baseline/run.json'; - const source = documents.get(runPath)!; - const serving = { - ...Object.fromEntries(entries(at(source, 'serving'))), - observed_batch_sizes: [1, 1, 1, 1], - observed_replica_ids: [0], - queue_delay_seconds: { - values: [0, 1, 2, 3], - sample_count: 4, - valid_clip_count: 4, - missing_count: 0, - p50: 1.5, - p90: null, - p95: null, - }, - }; - documents.set(runPath, { ...Object.fromEntries(entries(source)), serving }); - const matrixDoc = documents.get('serving-smoke.json')!; - documents.set('serving-smoke.json', { - ...Object.fromEntries(entries(matrixDoc)), - cells: rows(at(matrixDoc, 'cells')).map((cell, index) => - index - ? cell - : { - ...Object.fromEntries(entries(cell)), - metrics: { ...Object.fromEntries(entries(at(cell, 'metrics'))), serving }, - }, - ), - }); - cy.mount( - - undefined} /> - , - ); - cy.contains('dt', 'Observed batch sizes').next().should('have.text', '1'); - cy.contains('dt', 'Observed replica IDs').next().should('have.text', '0'); - cy.get('[data-testid="tradeoff-server-timing"]').within(() => { - cy.get('summary').click(); - cy.contains('dt', 'Queue delay') - .next() - .should('contain', '1.5 / — / —') - .and('contain', '4 / 4 / 0'); - }); - cy.get('[aria-label="Latency axis · lower is better"]').should( - 'contain', - 'Median client-ready latency (s)', - ); - cy.get('circle.point').should('have.length', 3); - }); - - it('defaults to P90 when individual serving cells have enough samples', () => { - const formal = matrix(); - formal.bundle = { - ...servingFixture('123', 'NVIDIA H200', 20), - result: null, - manifestSha256: 'synthetic-formal', - }; - cy.mount( - - undefined} /> - , - ); - cy.get('[role="combobox"][aria-label="Latency axis · lower is better"]').should( - 'contain', - 'P90 client-ready latency (s)', - ); - cy.get('circle.point').should('have.length', 3); - cy.get('tbody tr') - .should('have.length', 3) - .each((row) => { - cy.wrap(row).should('contain', '20 / 20 / 20'); - }); - }); - it('shows three labeled cell medians and opens the selected concurrency result', () => { - const open = cy.stub().as('openCell'); - cy.mount( - - - , - ); - cy.get('[role="combobox"][aria-label="Latency axis · lower is better"]').should( - 'contain', - 'Median client-ready latency (s)', - ); - cy.get('circle.point').should('have.length', 3); - cy.get('path.line-path') - .should('have.length', 1) - .invoke('attr', 'd') - .should('match', /^M[^L]+L[^L]+L[^L]+$/u); - cy.contains('Lines connect measured concurrency settings').should('be.visible'); - cy.get('path.line-path') - .invoke('attr', 'd') - .then((before) => { - cy.get('[data-testid="video-tradeoff-chart"] svg').trigger('wheel', { - deltaY: -200, - shiftKey: true, - clientX: 400, - clientY: 300, - force: true, - }); - cy.get('path.line-path').invoke('attr', 'd').should('not.equal', before); - }); - cy.get('text.serving-point-label') - .should('have.length', 3) - .then((labels) => { - expect([...labels].map((label) => label.textContent)).to.deep.equal(['C1', 'C2', 'C4']); - }); - cy.contains('Closed-loop smoke · serving capacity not qualified').should('be.visible'); - cy.get('tbody tr') - .should('have.length', 3) - .each((row) => { - cy.wrap(row).should('contain', '4 / 4 / 4').and('contain', '15'); - }); - cy.get('[role="combobox"][aria-label="Latency axis · lower is better"]').click(); - cy.contains('[role="option"]', 'P90 client-ready latency (s)').click(); - cy.get('circle.point').should('not.exist'); - cy.get('path.line-path').should('not.exist'); - cy.contains('No points qualify for these axes yet.').should('be.visible'); - cy.contains('button', 'Show cell medians').click(); - cy.get('circle.point').should('have.length', 3); - cy.get('path.line-path').should('have.length', 1); - cy.contains('tbody button', 'Client concurrency 4').click(); - cy.contains('button', 'Open videos and full result').click(); - cy.get('@openCell') - .should('have.been.calledOnce') - .its('firstCall.args.0.cellId') - .should('equal', 'c4'); - }); - it('shows the Chinese serving labels with per-cell medians', () => { - cy.mount( - - undefined} /> - , - ); - cy.get('[role="combobox"][aria-label="延迟轴 · 越低越好"]').should( - 'contain', - '客户端就绪延迟中位数(秒)', - ); - cy.get('circle.point').should('have.length', 3); - cy.get('path.line-path').should('have.length', 1); - cy.contains('各并发数的实测结果').should('be.visible'); - cy.contains('客户端并发数 2').should('be.visible'); - cy.contains('闭环冒烟测试 · 服务容量未经验证').should('be.visible'); - }); -}); diff --git a/packages/app/cypress/e2e/navigation.cy.ts b/packages/app/cypress/e2e/navigation.cy.ts index 9f032c21d..2ff0acffa 100644 --- a/packages/app/cypress/e2e/navigation.cy.ts +++ b/packages/app/cypress/e2e/navigation.cy.ts @@ -1,6 +1,7 @@ import { SUPPLEMENTAL_BENCHMARK_ROWS } from '../../src/lib/benchmarks/supplemental-benchmarks'; import { OVERLAY_RUN_ID, OVERLAY_RUN_URL } from '../support/overlay-fixtures'; -import { servingArtifact, videoRun } from '../support/video-artifacts'; +import { videoHistoryEntry } from '../../src/components/video-benchmark/history'; +import { servingArtifact } from '../support/video-artifacts'; const datum = (el: Element) => (el as Element & { __data__: { x: number; y: number } }).__data__; @@ -370,91 +371,46 @@ describe('TPUv7 results', { testIsolation: true }, () => { }); }); -describe('H3 video artifact viewer', () => { +describe('H3 video performance history', () => { beforeEach(() => { - cy.intercept('GET', '/api/video-runs?page=*', { runs: [], nextPage: null }); - }); - it('shares a comparison built by opening two CI runs through the page', () => { - for (const saved of [ - servingArtifact(123, 40, 'NVIDIA H200'), - servingArtifact(456, 41, 'NVIDIA B200'), - ]) { - cy.intercept('GET', `/api/video-runs?run=${saved.runId}`, { - run: videoRun(Number(saved.runId), 'success'), - artifacts: [saved.artifact], - }); - cy.intercept( - 'GET', - `/api/video-runs?run=${saved.runId}&artifact=${saved.artifact.id}&format=media`, - saved, - ); - } - cy.intercept('GET', 'https://media.test/**', { statusCode: 204 }); - cy.visit('/video?view=tradeoff&run=123&artifact=40'); - cy.get('[data-testid="video-tradeoff"] tbody tr').should('have.length', 3); - cy.contains('summary', 'Run details and artifact selection').click(); - cy.get('[data-testid="video-ci-runs"] form input').type('456'); - cy.get('[data-testid="video-ci-runs"] form').submit(); - cy.get('[data-testid="video-tradeoff"] tbody tr').should('have.length', 6); - cy.location().should((location) => { - const params = new URLSearchParams(location.search); - expect(params.get('run')).to.equal('456'); - expect(params.get('compare')).to.equal('123.40,456.41'); + cy.intercept('GET', '/api/video-runs?format=history&page=*', { + schemaVersion: 1, + entries: [], + nextPage: null, }); - cy.reload(); - cy.get('[data-testid="video-tradeoff"] tbody tr').should('have.length', 6); - cy.get('[data-testid="video-tradeoff"]') - .should('contain', 'NVIDIA H200') - .and('contain', 'NVIDIA B200'); }); - it('restores a cross-hardware comparison on reload and retains good results when one artifact fails', () => { - const h200 = servingArtifact(123, 40, 'NVIDIA H200'); - const b200 = servingArtifact(456, 41, 'NVIDIA B200'); - for (const saved of [h200, b200]) { - cy.intercept('GET', `/api/video-runs?run=${saved.runId}`, { - run: videoRun(Number(saved.runId), 'success'), - artifacts: [saved.artifact], - }); - cy.intercept( - 'GET', - `/api/video-runs?run=${saved.runId}&artifact=${saved.artifact.id}&format=media`, - saved, - ); - } - cy.intercept('GET', '/api/video-runs?run=789&artifact=42&format=media', { - statusCode: 503, - body: { error: 'Synthetic unavailable artifact' }, - }); - cy.intercept('GET', 'https://media.test/**', { statusCode: 204 }); - cy.visit('/video?view=tradeoff&run=123&artifact=40&source=123&compare=123.40,456.41,789.42'); - cy.get('[data-testid="video-tradeoff"] tbody tr').should('have.length', 6); - cy.get('[data-testid="tradeoff-detail"]').should('contain', 'NVIDIA H200'); - cy.get('[data-testid="video-tradeoff"]') - .should('contain', 'NVIDIA H200') - .and('contain', 'NVIDIA B200'); - cy.get('[role="alert"]').should('contain', 'Some comparison results could not be loaded'); + it('opens published history behind the dashboard and restores its filters after reload', () => { + const entry = videoHistoryEntry(servingArtifact(), '2026-09-09T00:00:00Z'); + cy.intercept('GET', '/api/video-runs?format=history&page=1', { + schemaVersion: 1, + entries: [entry], + nextPage: null, + }).as('history'); + cy.visit('/video'); + cy.wait('@history'); + cy.get('[data-testid="video-history"]').should('not.exist'); + cy.get('[data-testid="video-history-section"] summary').click(); + cy.get('[data-testid="video-history"] h1').should('contain', 'Performance history'); + cy.get('[data-testid="video-history-observation"]').should('have.length', 3); + cy.get('[aria-label="Client concurrency"]').click(); + cy.contains('[role="option"]', 'C2').click(); + cy.get('[data-testid="video-history-observation"]') + .should('have.length', 1) + .and('contain', 'C2'); cy.reload(); - cy.get('[data-testid="video-tradeoff"] tbody tr').should('have.length', 6); - cy.get('[data-testid="tradeoff-detail"]').should('contain', 'NVIDIA H200'); - cy.contains( - '[data-testid="video-tradeoff"] tbody button', - /NVIDIA B200.*Client concurrency 4/, - ).click(); - cy.contains('button', 'Open videos and full result').click(); - cy.get('[data-testid="serving-selected-metrics"]').should('contain', 'C4'); - cy.location('search').should('include', 'run=456').and('include', 'compare='); - cy.get('[data-testid="serving-media"] video') - .should('have.attr', 'src') - .and('include', '/gpu/c4/'); + cy.get('[data-testid="video-history-observation"]') + .should('have.length', 1) + .and('contain', 'C2') + .and('contain', '240 / —'); + cy.location('search').should('contain', 'history-concurrency=2'); }); - it('uses the shared unlock for navigation and keeps an empty viewer free of sample results', () => { + it('keeps the hidden Video tab behind the shared unlock and noindex', () => { cy.viewport(1440, 1000); cy.visit('/video', { onBeforeLoad(win) { win.localStorage.removeItem('inferencex-feature-gate'); }, }); - cy.get('[data-testid="video-ci-runs"]').should('contain', 'No H3 runs in this page'); cy.get('video[data-role]').should('not.exist'); cy.get('[data-testid="tab-trigger-hidden"]').should('not.exist'); cy.get('body').type('{upArrow}{upArrow}{downArrow}{downArrow}'); @@ -462,16 +418,10 @@ describe('H3 video artifact viewer', () => { cy.contains('a', 'Video').should('have.attr', 'href', '/video'); cy.get('head meta[name="robots"]').should('have.attr', 'content', 'noindex, nofollow'); }); - it('shows a recoverable load error and the Chinese empty state', () => { - cy.visit('/video'); - cy.contains('summary', 'Local artifact tools').click(); - cy.get('[data-testid="video-benchmark"]').contains('summary', 'Manifest URL').click(); - cy.get('input[aria-label="Manifest URL"]').type('https://example.com/wrong.json'); - cy.contains('button', 'Load manifest').click(); - cy.get('[role="alert"]').should('contain', 'Could not load this bundle'); - cy.get('video[data-role]').should('not.exist'); - cy.visit('/zh/video'); - cy.get('[data-testid="video-ci-runs"]').should('contain', '本页 GitHub 历史中没有 H3 运行'); + it('renders the Chinese history section for a history deep link', () => { + cy.visit('/zh/video?history-hardware=H200'); + cy.get('[data-testid="video-history-section"]').should('have.attr', 'open'); + cy.get('[data-testid="video-history"] h1').should('contain', '性能历史'); cy.get('head meta[name="robots"]').should('have.attr', 'content', 'noindex, nofollow'); }); }); diff --git a/packages/app/cypress/e2e/video-dashboard.cy.ts b/packages/app/cypress/e2e/video-dashboard.cy.ts new file mode 100644 index 000000000..58b1bd553 --- /dev/null +++ b/packages/app/cypress/e2e/video-dashboard.cy.ts @@ -0,0 +1,389 @@ +// Runs against the E2E_FIXTURES=1 server, which serves the retained +// H100/H200/B200 C1/C2/C4 observations from cypress/fixtures/api/video-history.json. +// Lead (C1) cells, per participating GPU: H100 5.37 videos/GPU-hr at $1.17/GPU-hr, +// H200 5.97 at $1.22, B200 11.53 at $1.73. Every clip is 8 s, so the $0.080/video-s +// API reference lists at $0.640 per video. +describe('Video hardware dashboard (E2E fixtures)', () => { + beforeEach(() => { + // The Compare panel fetches two published artifacts once in view; keep the spec offline. + cy.intercept('GET', '/api/video-runs?run=*', { statusCode: 204 }).as('media'); + }); + it('exports visible measured points as CSV and PNG and excludes unqualified quality rows', () => { + let capturedCsv: Blob | undefined; + let capturedPng: string | undefined; + const exportCsv = () => { + cy.then(() => { + capturedCsv = undefined; + }); + cy.get('[data-testid="video-dashboard"] [data-testid="export-button"]').first().click(); + cy.get('[data-testid="export-csv-button"]').click(); + return cy.then(() => { + expect(capturedCsv, 'CSV created by the dashboard export button').not.to.equal(undefined); + return capturedCsv!.text(); + }); + }; + cy.visit('/video?v_y=videosPerGpuHour', { + onBeforeLoad(win) { + win.URL.createObjectURL = (object: Blob | MediaSource) => { + if (object instanceof win.Blob) capturedCsv = object; + return 'blob:video-csv-export-test'; + }; + win.HTMLAnchorElement.prototype.click = function (this: HTMLAnchorElement) { + if (this.download.endsWith('.png')) capturedPng = this.href; + }; + }, + }); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 3); + cy.contains('[data-testid="video-legend"] li', 'H100').click(); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 2); + exportCsv().then((csv) => { + const [header, ...rows] = csv.trimEnd().split('\n'); + expect(rows, 'one exported row for each visible point').to.have.length(2); + expect(header.split(',')).to.include.members([ + 'id', + 'hardware', + 'source_id', + 'source_sha', + 'manifest_sha256', + 'artifact_id', + 'ci_run', + ]); + expect(rows.map((row) => row.split(',')[1])).to.deep.equal(['B200', 'H200']); + expect(csv).not.to.include(',H100,'); + expect(rows[0]).to.include( + 'https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34341996789', + ); + expect(rows[1]).to.include( + 'https://github.com/SemiAnalysisAI/InferenceX/actions/runs/34342452354', + ); + for (const row of rows) expect(row.split(',')[0]).to.match(/^[a-f0-9]{64}:c1$/u); + }); + cy.get('[data-testid="video-dashboard"] [data-testid="export-button"]').first().click(); + cy.get('[data-testid="export-png-button"]').click(); + cy.wrap(null).should(() => { + expect(capturedPng, 'PNG generated by the shared chart export hook').to.match( + /^data:image\/png;base64,/u, + ); + const bytes = atob(capturedPng!.split(',')[1]); + expect(bytes.length, 'nonempty encoded PNG').to.be.greaterThan(128); + expect(bytes.slice(0, 8), 'PNG file signature').to.equal('\u0089PNG\r\n\u001A\n'); + }); + cy.get('[aria-label="Quality threshold (minimum)"]').click(); + cy.contains('[role="option"]', '≥ 4 / 4').click(); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('not.exist'); + exportCsv().then((csv) => { + expect(csv.trimEnd().split('\n'), 'unjudged quality exports headers only').to.have.length(1); + expect(csv).to.include('quality_dimension').and.include('reader_quality_threshold'); + expect(csv).not.to.include('https://github.com/SemiAnalysisAI/InferenceX/actions/runs/'); + }); + }); + it('keeps hardware visibility after reload and on a shared Chinese URL', () => { + cy.visit('/video'); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 3); + cy.contains('[data-testid="video-legend"] li', 'H100').click(); + cy.location('search').should('contain', 'v_hidden=h100'); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 2); + cy.reload(); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 2); + cy.location('search').then((search) => cy.visit(`/zh/video${search}&v_view=table`)); + cy.get('[data-testid="video-points-table"] tbody tr').should('have.length', 2); + cy.get('[data-testid="video-points-table"]').should('not.contain', 'H100'); + cy.contains('[data-testid="video-legend"] li', 'H100').click(); + cy.get('[data-testid="video-points-table"] tbody tr').should('have.length', 3); + cy.location('search').should('not.contain', 'v_hidden='); + }); + it('leads with the cross-hardware chart and restores v_ params from the URL', () => { + cy.visit('/video?v_y=kjPerVideo&v_tier=r'); + // One measured deployment per hardware: three points, no frontier line, and the + // queued C2/C4 cells never reach the chart. + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 3); + cy.get('[data-testid="video-hardware-chart"] path.line-path').should('have.length', 0); + cy.get('[data-testid="video-chart-caption"]').should( + 'have.text', + 'Measured observations are shown as separate points (4 GPU · TP2 × Ulysses 2). A frontier line requires distinct measured deployments for the same hardware within a comparable cohort.', + ); + cy.get('[data-testid="video-chart-card"]') + .should('contain', 'GPU-board energy per video (kJ)') + .and('contain', 'Rent - 3 Year Commit'); + cy.get('[data-testid="video-idle-note"]').should( + 'have.text', + 'Per participating GPU: B200 (4 of 8), H100 (4 of 8) reserved more boards than one video uses; the idle boards are not counted.', + ); + cy.get('[data-testid="video-tco-badge"]').first().should('contain', 'B200'); + cy.get('[data-testid="video-replay"]').should('be.visible'); + // Model is selectable; workload and deployment remain measured facts. + cy.get('[data-testid="video-config-fact"]').should('have.length', 2); + cy.get('[aria-label="Model"]').should('contain', 'MiniMax-H3'); + cy.get('[data-testid="video-config-fact"]') + .eq(0) + .should('contain', 'Workload') + .and('contain', '1344 × 768 · 8 s · 24 fps · 50 steps'); + cy.get('[data-testid="video-config-fact"]') + .eq(1) + .should('contain', 'Deployment') + .and('contain', '4 GPU · TP2 × Ulysses 2'); + cy.get('[data-testid="video-config-bar"] [role="combobox"]').should('have.length', 5); + cy.get('[aria-label="X-axis metric"]').should('contain', 'P90 time to video (s)'); + cy.get('[aria-label="Y-axis metric"]').should('contain', 'GPU-board energy per video (kJ)'); + // The cost tier is picked in the chart caption, as on /inference. + cy.get('[data-testid="video-cost-tier"]').should('contain', 'Rent - 3 Year Commit'); + cy.get('[data-testid="video-config-bar"]').should('not.contain', 'GPU basis'); + cy.get('[data-testid="video-legend"]').should('contain', 'MI355X · not measured'); + cy.get('[data-testid="video-legend"] [role="switch"]').should('have.length', 1); + cy.get('[data-testid="video-optimal-only"]').should('have.attr', 'aria-checked', 'true'); + cy.get('[data-testid="video-dashboard"] h1').should('contain', 'VideoGenX · MiniMax-H3'); + cy.get('[data-testid="share-button"]').should('be.visible'); + cy.get('[data-testid="video-compare-toggle"]').should('have.attr', 'aria-expanded', 'false'); + cy.get('[data-testid="video-evidence-toggle"]').should('have.attr', 'aria-expanded', 'false'); + // KPI cards read each hardware's lead deployment at the selected tier (Rent - 3 Year Commit: + // $2 / $2.9 / $3.7 per GPU-hour); the API list price sits beside the TCO cost. + cy.get('[data-testid="video-kpi-card"]').should('have.length', 4); + cy.get('[data-testid="video-kpi-card"][data-hardware="b200"]') + .should('contain', '4 of 8 GPUs · TP2 × Ulysses 2') + .and('contain', 'P50 time to video (s)') + .and('contain', '77.9') + .and('contain', 'P90 78.3') + .and('contain', 'Videos / GPU-hr') + .and('contain', '11.53') + .and('contain', 'TCO / video') + .and('contain', '$0.321') + .and('contain', 'API list $0.640') + .and('contain', 'kJ / video') + .and('contain', '301.1') + .and('not.contain', 'Profit'); + cy.get('[data-testid="video-kpi-card"][data-hardware="h200"]') + .should('contain', '4 of 4 GPUs') + .and('contain', '$0.485'); + cy.get('[data-testid="video-kpi-card"][data-hardware="h100"]') + .should('contain', '4 of 8 GPUs') + .and('contain', '$0.372'); + cy.get('[data-testid="video-history-section"]').should('not.have.attr', 'open'); + cy.get('[data-testid="video-history"]').should('not.exist'); + }); + it('lists only the measured deployments in the table view, without a concurrency column', () => { + cy.visit('/video?v_view=table'); + cy.get('[data-testid="video-hardware-chart"]').should('not.exist'); + // One row per hardware: the queued C2/C4 cells stay out, so the C4-only H100 run never appears. + cy.get('[data-testid="video-points-table"] tbody tr').should('have.length', 3); + cy.get('[data-testid="video-points-table"]').should('not.contain', '#34344378350'); + cy.get('[data-testid="data-table-preset-all"]').click(); + cy.get('[data-testid="video-points-table"] thead th').should(($headers) => { + expect([...$headers].map((th) => th.textContent?.trim())).to.deep.equal([ + 'Hardware', + 'Deployment', + 'Valid / scheduled', + 'P50 time to video (s)', + 'P90 time to video (s)', + 'Videos per GPU-hour', + 'Videos per $1 TCO (Owning at Large Hyperscaler Volume)', + 'TCO cost per video (Owning at Large Hyperscaler Volume)', + 'API list price per video', + 'GPU-board energy per video (kJ)', + 'Mean board power / enforced limit (%)', + 'Prompt adherence (0–4)', + 'Board power (W)', + 'CI run', + 'Details', + ]); + }); + cy.contains('[data-testid="video-points-table"] tbody tr', 'B200') + .should('contain', '4 GPU · TP2 × Ulysses 2') + .and('contain', '20 / 20') + .and('contain', '77.9') + .and('contain', '78.3') + .and('contain', '11.53') + .and('contain', '6.66') + .and('contain', '$0.150') + .and('contain', '$0.640') + .and('contain', '301.1') + .and('contain', '96.4') + .and('contain', '3,856') + .and('contain', '#34341996789'); + cy.contains('[data-testid="video-points-table"] tbody tr', 'H200') + .should('contain', '150.6') + .and('contain', '5.97') + .and('contain', '4.9') + .and('contain', '$0.204') + .and('contain', '411') + .and('contain', '97.4') + .and('contain', '2,727') + .and('contain', '#34342452354'); + cy.contains('[data-testid="video-points-table"] tbody tr', 'H100') + .should('contain', '167.3') + .and('contain', '5.37') + .and('contain', '4.59') + .and('contain', '$0.218') + .and('contain', '431.1') + .and('contain', '91.9') + .and('contain', '2,575') + .and('contain', '#34336635063'); + }); + it('prices the API list reference from v_api and resets it to the dated default', () => { + cy.visit('/video?v_api=0.05'); + cy.get('[data-testid="video-api-price"]').should('have.value', '0.05'); + cy.get('[data-testid="video-chart-card"]').should('contain', 'API reference: $0.050/video-s'); + // 8 s clips: $0.05/video-s lists at $0.400 per video on every hardware. + cy.get('[data-testid="video-kpi-card"][data-hardware="b200"]').should( + 'contain', + 'API list $0.400', + ); + cy.get('[data-testid="video-kpi-card"][data-hardware="h100"]').should( + 'contain', + 'API list $0.400', + ); + cy.get('[data-testid="video-api-price-reset"]').click(); + cy.get('[data-testid="video-api-price"]').should('have.value', '0.08'); + cy.get('[data-testid="video-kpi-card"][data-hardware="b200"]').should( + 'contain', + 'API list $0.640', + ); + cy.location('search').should('not.contain', 'v_api'); + // A non-positive price never prices at 0: the dated reference stays in force. + cy.visit('/video?v_api=0'); + cy.get('[data-testid="video-api-price"]').should('have.value', '0.08'); + cy.get('[data-testid="video-kpi-card"][data-hardware="h200"]').should( + 'contain', + 'API list $0.640', + ); + cy.get('[data-testid="video-api-reference-caption"]').should('contain', 'captured 2026-09-24'); + }); + it('compares the slowest lead deployment against the fastest with time, cost and energy deltas', () => { + cy.visit('/video'); + cy.get('[data-testid="video-compare-toggle"]').click(); + cy.get('[data-testid="video-compare"]').scrollIntoView(); + cy.wait('@media'); + // Default pair: slowest C1 P50 (H100) as baseline, fastest (B200) as candidate. Two hardware + // selects and nothing else to pick: no case dropdown, no blind mode. + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'H100'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'B200'); + cy.get('[data-testid="video-compare"] [role="combobox"]').should('have.length', 2); + cy.get('[data-testid="video-compare-clips"]').should( + 'contain', + 'Clips are not published for one of these runs', + ); + cy.get('[data-testid="video-compare-table"] tbody tr').should(($rows) => { + expect([...$rows].map((row) => row.dataset.metric)).to.deep.equal([ + 'p50Latency', + 'dollarsPerVideo', + 'kjPerVideo', + ]); + }); + cy.get('[data-testid="video-compare-table"] tr[data-metric="p50Latency"]') + .should('contain', '167.3') + .and('contain', '77.9') + .and('contain', '-53.4%') + .and('contain', '×0.47') + .find('[data-tone]') + .should('have.attr', 'data-tone', 'better'); + cy.get('[data-testid="video-compare-table"] tr[data-metric="dollarsPerVideo"]') + .should('contain', '$0.218') + .and('contain', '$0.150') + .and('contain', '-31.1%') + .find('[data-tone]') + .should('have.attr', 'data-tone', 'better'); + cy.get('[data-testid="video-compare-table"] tr[data-metric="kjPerVideo"]') + .should('contain', '431.1') + .and('contain', '301.1') + .and('contain', '-30.2%') + .find('[data-tone]') + .should('have.attr', 'data-tone', 'better'); + cy.get('[data-testid="video-compare"]').should( + 'contain', + 'Cost tier: Owning at Large Hyperscaler Volume.', + ); + cy.get('[data-testid="video-compare-swap"]').click(); + cy.get('[data-testid="video-compare-baseline"]').should('contain', 'B200'); + cy.get('[data-testid="video-compare-candidate"]').should('contain', 'H100'); + cy.get('[data-testid="video-compare-table"] tr[data-metric="p50Latency"]') + .should('contain', '+114.7%') + .find('[data-tone]') + .should('have.attr', 'data-tone', 'worse'); + cy.location('search').should('contain', 'v_base=b200').and('contain', 'v_cand=h100'); + }); + it('opens the history section for history deep links and renders the Chinese dashboard', () => { + cy.visit('/video?history-hardware=H200'); + cy.get('[data-testid="video-history-section"]').should('have.attr', 'open'); + cy.get('[data-testid="video-history"] h1').should('contain', 'Performance history'); + cy.visit('/zh/video'); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 3); + cy.get('[data-testid="video-chart-card"]').should( + 'contain', + '每 1 美元 TCO 生成视频数(Hyperscaler 自有设备)', + ); + cy.get('[data-testid="video-idle-note"]').should( + 'have.text', + '按参与计算的 GPU 计:B200(4 / 8 张)、H100(4 / 8 张)预留的板卡多于单条视频所需,空闲板卡未计入。', + ); + cy.get('[data-testid="video-config-bar"]') + .should('contain', '模型') + .and('contain', '工作负载') + .and('contain', '4 张 GPU · TP2 × Ulysses 2') + .and('contain', 'X 轴指标'); + cy.get('[data-testid="video-cost-tier"]').should('contain', 'Hyperscaler 自有设备'); + cy.get('[data-testid="video-legend"]').should('contain', '仅最优'); + cy.get('[data-testid="video-kpi-card"][data-hardware="b200"]') + .should('contain', 'TCO / 条视频') + .and('contain', '$0.150') + .and('contain', 'API 标价 $0.640'); + cy.get('[data-testid="video-kpi-card"][data-hardware="mi355x"]').should('contain', '未测得'); + }); + it('fits a phone viewport without horizontal page scroll in both locales', () => { + cy.viewport(390, 844); + for (const path of ['/video', '/zh/video']) { + cy.visit(path); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 3); + cy.get('[data-testid="video-kpi-card"]').should('have.length', 4); + cy.get('[data-testid="video-engine-coverage"] [data-model][data-engine]').should( + 'have.length', + 4, + ); + cy.document().then((doc) => { + expect(doc.documentElement.scrollWidth, `${path} page width`).to.be.at.most( + doc.documentElement.clientWidth, + ); + }); + cy.screenshot(`video-dashboard-390${path.startsWith('/zh') ? '-zh' : '-en'}`, { + capture: 'fullPage', + overwrite: true, + }); + } + }); +}); + +describe('Video model selection', () => { + it('switches models without leaking H3 results, survives reload and fits mobile', () => { + cy.visit('/video?v_api=0.12&v_hidden=h100'); + cy.get('[aria-label="Model"]').click(); + cy.contains('[role="option"]', 'Wan2.2-T2V-A14B').click(); + cy.location('search') + .should('contain', 'v_model=wan22') + .and('not.contain', 'v_api') + .and('not.contain', 'v_hidden'); + cy.get('[data-testid="video-model-empty"]').should('be.visible'); + cy.get('[data-testid="video-dashboard"] [data-testid="export-button"]').first().click(); + cy.get('[data-testid="export-png-button"]').should('have.attr', 'aria-disabled', 'true'); + cy.get('[data-testid="export-csv-button"]').should('be.visible'); + cy.get('body').type('{esc}'); + cy.get('[data-testid="video-api-price"]').should('have.value', ''); + cy.get('[data-testid="video-kpi-card"]').should('not.exist'); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('not.exist'); + cy.get('[data-testid="video-history-section"] summary').click(); + cy.get('[data-testid="video-history"]').should('contain', 'No matching entries'); + cy.reload(); + cy.get('[aria-label="Model"]').should('contain', 'Wan2.2'); + cy.get('[data-testid="video-model-empty"]').should('be.visible'); + cy.get('[data-testid="video-api-price"]').type('0.05').blur(); + cy.location('search').should('contain', 'v_api=0.05'); + cy.get('[data-testid="video-api-price-reset"]').click(); + cy.get('[data-testid="video-api-price"]').should('have.value', ''); + cy.viewport(390, 844); + cy.get('[data-testid="video-model-empty"]').should('be.visible'); + cy.document().then((doc) => expect(doc.documentElement.scrollWidth).to.be.at.most(390)); + cy.get('[aria-label="Model"]').click(); + cy.contains('[role="option"]', 'MiniMax-H3').click(); + cy.get('[data-testid="video-api-price"]').should('have.value', '0.08'); + cy.get('[data-testid="video-hardware-chart"] circle.point').should('have.length', 3); + cy.visit('/zh/video?v_model=wan22'); + cy.get('[data-testid="video-model-empty"]').should('contain', '暂无 Wan2.2'); + }); +}); diff --git a/packages/app/cypress/fixtures/api/_manifest.json b/packages/app/cypress/fixtures/api/_manifest.json index d86edd3b9..7f573b42d 100644 --- a/packages/app/cypress/fixtures/api/_manifest.json +++ b/packages/app/cypress/fixtures/api/_manifest.json @@ -1,6 +1,6 @@ { "schemaVersion": 1, - "generatedAt": "2026-10-16T02:59:38.939Z", + "generatedAt": "2026-10-17T05:15:55.279Z", "fixtures": { "agentic-catalog": { "bytes": 10027, @@ -99,6 +99,13 @@ "sha256": "f28d2220e5246ad70113e38e49394bc7a0bfe9afba8cedf08b90e39042e53357", "source": null, "topLevel": "object" + }, + "video-history": { + "bytes": 108039, + "capturedAt": "2026-10-17T05:15:55.279Z", + "sha256": "f27fbbcd2c6a8154b030939714501b8480b59fd3b0d5701007cd2e29e2a1559e", + "source": "Retained public H3 metadata replay from original runs via videoHistoryEntry (canonical workload identity, per-cell GPU counts, board power and server layout); publication time unavailable. No new benchmark or publication.", + "topLevel": "object" } } } diff --git a/packages/app/cypress/fixtures/api/video-history.json b/packages/app/cypress/fixtures/api/video-history.json new file mode 100644 index 000000000..dc973402f --- /dev/null +++ b/packages/app/cypress/fixtures/api/video-history.json @@ -0,0 +1,617 @@ +{ + "schemaVersion": 1, + "entries": [ + { + "id": "34336635063.10103071687", + "runId": "34336635063", + "artifact": { + "id": 10103071687, + "name": "h3-video-34336635063-1", + "expired": false, + "size_in_bytes": 118562850, + "digest": "sha256:575c0016fa96f9fa080706285611509db6f2d72be5f8ffd0248cda3b91f2de42", + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "34336635063", + "sha256": "7df99e50c015669f4dd9ef62549317870f197cfe0fc5d163cf8bf0fb3b88dd5e", + "sourceSha": "25de33c57f5c2ac0189c3f3a8de396057266dfb9", + "hardware": "H100", + "execution": "complete", + "observedAt": "2026-09-09T09:47:33.704519+00:00", + "kind": "observation", + "observations": [ + { + "id": "7df99e50c015669f4dd9ef62549317870f197cfe0fc5d163cf8bf0fb3b88dd5e:c1", + "cell": "c1", + "hardware": "NVIDIA H100 80GB HBM3", + "concurrency": 1, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 167.33339739358053, + "p90": 168.11354550393298, + "clipsGpuHour": 5.3741390883412175, + "energyKj": 431.14963489170344, + "participating": 4, + "allocated": 8, + "replicas": null, + "wallSeconds": 3349.37367718108, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 2574.550733165641, + "enforcedLimitW": 2800, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + }, + { + "id": "7df99e50c015669f4dd9ef62549317870f197cfe0fc5d163cf8bf0fb3b88dd5e:c2", + "cell": "c2", + "hardware": "NVIDIA H100 80GB HBM3", + "concurrency": 2, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 334.78318604291417, + "p90": 336.9140758761205, + "clipsGpuHour": 5.368742995784034, + "energyKj": 431.32628015924786, + "participating": 4, + "allocated": 8, + "replicas": null, + "wallSeconds": 3352.7401133067906, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 2573.016825749808, + "enforcedLimitW": 2800, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + } + ], + "fidelity": null, + "calibration": null, + "releaseQualified": false, + "error": null + } + ], + "error": null + }, + { + "id": "34344378350.10106004321", + "runId": "34344378350", + "artifact": { + "id": 10106004321, + "name": "h3-video-34344378350-1", + "expired": false, + "size_in_bytes": 61499259, + "digest": "sha256:a0bd28cc430c443c1be086bb33f16636a04e51acb0d92f554228e1f1b7409329", + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "34344378350", + "sha256": "98206c73d05226c162cad14f456d7671640cbd1d090945f456f5427713fdde7f", + "sourceSha": "b04b09c505cfec39f71a7971b28c8af6c93882c9", + "hardware": "H100", + "execution": "complete", + "observedAt": "2026-09-09T12:11:43.125736+00:00", + "kind": "observation", + "observations": [ + { + "id": "98206c73d05226c162cad14f456d7671640cbd1d090945f456f5427713fdde7f:c4", + "cell": "c4", + "hardware": "NVIDIA H100 80GB HBM3", + "concurrency": 4, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 669.2438137787394, + "p90": 673.7058754772879, + "clipsGpuHour": 5.368345559881099, + "energyKj": 430.80324425396446, + "participating": 4, + "allocated": 8, + "replicas": null, + "wallSeconds": 3352.988327450119, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 2569.709833308179, + "enforcedLimitW": 2800, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + } + ], + "fidelity": null, + "calibration": null, + "releaseQualified": false, + "error": null + } + ], + "error": null + }, + { + "id": "34342452354.10107476604", + "runId": "34342452354", + "artifact": { + "id": 10107476604, + "name": "h3-video-34342452354-1", + "expired": false, + "size_in_bytes": 175903798, + "digest": "sha256:ac85510c000f69098db37149ae56b48964563c7a1dc4b26a31f71317b8b3599a", + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "34342452354", + "sha256": "ad62f1b44954b9733cb513b7dd1d54c224759a20956d9a1f257b190ce8d4da35", + "sourceSha": "6ea0bf8750d97b2f89a913536dd0dc4aad5bf74e", + "hardware": "H200", + "execution": "complete", + "observedAt": "2026-09-09T10:52:07.062086+00:00", + "kind": "observation", + "observations": [ + { + "id": "ad62f1b44954b9733cb513b7dd1d54c224759a20956d9a1f257b190ce8d4da35:c1", + "cell": "c1", + "hardware": "NVIDIA H200", + "concurrency": 1, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 150.61379817902343, + "p90": 151.10486977093387, + "clipsGpuHour": 5.973235869122123, + "energyKj": 410.954300515549, + "participating": 4, + "allocated": 4, + "replicas": null, + "wallSeconds": 3013.44202613004, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 2727.484797810607, + "enforcedLimitW": 2800, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + }, + { + "id": "ad62f1b44954b9733cb513b7dd1d54c224759a20956d9a1f257b190ce8d4da35:c2", + "cell": "c2", + "hardware": "NVIDIA H200", + "concurrency": 2, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 300.09254947002046, + "p90": 300.1774324030848, + "clipsGpuHour": 5.997056099146397, + "energyKj": 409.91272507134966, + "participating": 4, + "allocated": 4, + "replicas": null, + "wallSeconds": 3001.4726729940157, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 2731.4196931869433, + "enforcedLimitW": 2800, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + }, + { + "id": "ad62f1b44954b9733cb513b7dd1d54c224759a20956d9a1f257b190ce8d4da35:c4", + "cell": "c4", + "hardware": "NVIDIA H200", + "concurrency": 4, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 601.2122122829896, + "p90": 601.8977232379839, + "clipsGpuHour": 5.986775861646954, + "energyKj": 413.58283766782114, + "participating": 4, + "allocated": 4, + "replicas": null, + "wallSeconds": 3006.626674519968, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 2751.151140021047, + "enforcedLimitW": 2800, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + } + ], + "fidelity": null, + "calibration": null, + "releaseQualified": false, + "error": null + } + ], + "error": null + }, + { + "id": "34341996789.10104690935", + "runId": "34341996789", + "artifact": { + "id": 10104690935, + "name": "h3-video-34341996789-1", + "expired": false, + "size_in_bytes": 163563283, + "digest": "sha256:6b05cafd4c014e02c904f65c0e0d2f3d9a5b541ddc4f030372ed08d4a588c002", + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "34341996789", + "sha256": "ab90da9236114cd13de833a364159b8f84e69060de888f3e813635d5b7b26b82", + "sourceSha": "6ea0bf8750d97b2f89a913536dd0dc4aad5bf74e", + "hardware": "B200", + "execution": "complete", + "observedAt": "2026-09-09T10:47:03.922884+00:00", + "kind": "observation", + "observations": [ + { + "id": "ab90da9236114cd13de833a364159b8f84e69060de888f3e813635d5b7b26b82:c1", + "cell": "c1", + "hardware": "NVIDIA B200", + "concurrency": 1, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 77.94495720259147, + "p90": 78.31597462110221, + "clipsGpuHour": 11.52605424609145, + "energyKj": 301.08754501100987, + "participating": 4, + "allocated": 8, + "replicas": null, + "wallSeconds": 1561.6792716470081, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 3856.016973742528, + "enforcedLimitW": 4000, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + }, + { + "id": "ab90da9236114cd13de833a364159b8f84e69060de888f3e813635d5b7b26b82:c2", + "cell": "c2", + "hardware": "NVIDIA B200", + "concurrency": 2, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 153.6016560649732, + "p90": 154.1683327689534, + "clipsGpuHour": 11.691563929313219, + "energyKj": 300.10551714629605, + "participating": 4, + "allocated": 8, + "replicas": null, + "wallSeconds": 1539.571618376067, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 3898.6355959505895, + "enforcedLimitW": 4000, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + }, + { + "id": "ab90da9236114cd13de833a364159b8f84e69060de888f3e813635d5b7b26b82:c4", + "cell": "c4", + "hardware": "NVIDIA B200", + "concurrency": 4, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 8 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11, 11, 29, 47, 29, 29, 29, 47, 83, 11, 47, 83, 47, 11, 11, 83, 83, 29, 83, 47 · A single continuous medium close-up of an adult presenter facing the camera in a", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"spoken-greeting-s11\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"bicycle-pass-s11\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s29\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s47\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"piano-notes-s29\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"drum-taps-s29\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"bicycle-pass-s29\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"water-pour-s47\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s83\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"piano-notes-s11\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s47\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"bicycle-pass-s83\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"bicycle-pass-s47\",\"prompt\":\"A single continuous eye-level shot of a cyclist riding from left to right across a paved park path, passing the camera and continuing into the distance. The bicycle wheels keep turning and the leaves move gently in the breeze. Tire sounds, wind, and distant birds are audible. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47},{\"case_id\":\"water-pour-s11\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"drum-taps-s11\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11},{\"case_id\":\"spoken-greeting-s83\",\"prompt\":\"A single continuous medium close-up of an adult presenter facing the camera in a quiet studio. The presenter smiles naturally and says clearly, \\\"Good morning. It is a beautiful day to try something new.\\\" Mouth movements follow the spoken sentence. The camera remains still. No background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s83\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"water-pour-s29\",\"prompt\":\"A single continuous close-up of a person slowly pouring water from a clear glass pitcher into a ceramic cup on a wooden table. The water stream, rising water level, and small ripples are visible. The pouring and gentle splashing are audible in a quiet kitchen. The camera remains still. No speech, music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":29},{\"case_id\":\"piano-notes-s83\",\"prompt\":\"A single continuous close-up of two human hands playing a short, slow melody on an acoustic piano. The fingers visibly press and release the keys, with each note ringing naturally in the room. The camera remains still. The piano is the only prominent sound. No speech, background music, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":83},{\"case_id\":\"drum-taps-s47\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":47}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":8,\"flow_shift\":12,\"fps\":24,\"frame_count\":192,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 20, + "completed": 20, + "scheduled": 20, + "failed": 0, + "samples": 20, + "p50": 307.28381997847464, + "p90": 308.79631440097, + "clipsGpuHour": 11.69172635792307, + "energyKj": 299.96858693604275, + "participating": 4, + "allocated": 8, + "replicas": null, + "wallSeconds": 1539.5502297059866, + "durationSeconds": 8, + "frameCount": 192, + "avgPowerW": 3896.903348609339, + "enforcedLimitW": 4000, + "server": { + "tp": 2, + "ulysses": 2, + "attention": null + } + } + ], + "fidelity": null, + "calibration": null, + "releaseQualified": false, + "error": null + } + ], + "error": null + }, + { + "id": "34410724145.10127189553", + "runId": "34410724145", + "artifact": { + "id": 10127189553, + "name": "h3-fidelity-34410724145-1", + "expired": false, + "size_in_bytes": 47838865, + "digest": "sha256:e8f364dae253274e954003fd06b177b17215381520bbc5924a10607839873479", + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "34410724145", + "sha256": "c40cc39acd26f046f84b307d7f771ed38f28d96ba21860d6679bba7ba3e7cf29", + "sourceSha": "df5e5cd434ef262cac029925d8782e4290ea6d16", + "hardware": "", + "execution": "", + "observedAt": null, + "kind": "fidelity", + "observations": [], + "fidelity": "fail", + "calibration": "uncalibrated", + "releaseQualified": false, + "error": null + } + ], + "error": null + }, + { + "id": "34410729828.10127190894", + "runId": "34410729828", + "artifact": { + "id": 10127190894, + "name": "h3-fidelity-34410729828-1", + "expired": false, + "size_in_bytes": 47585397, + "digest": "sha256:bb701c77431bb999181d74cb99e4d957da13a6ff45ba927c3f89cf5add49ef69", + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "34410729828", + "sha256": "c2f6a036328e4b90fa1b364b16ca10353baa7eb2fef2d71571a1ab1465fc5382", + "sourceSha": "df5e5cd434ef262cac029925d8782e4290ea6d16", + "hardware": "", + "execution": "", + "observedAt": null, + "kind": "fidelity", + "observations": [], + "fidelity": "fail", + "calibration": "uncalibrated", + "releaseQualified": false, + "error": null + } + ], + "error": null + }, + { + "id": "39002901777.12090362113", + "runId": "39002901777", + "artifact": { + "id": 12090362113, + "name": "h3-results-39002901777-1", + "expired": false, + "size_in_bytes": 9285861, + "stored": true + }, + "publishedAt": null, + "sources": [ + { + "id": "39002901777", + "sha256": "a17baa4e13b3f1de1f70796e892d411a6efd2bcb5314b55d3e2c01d868a4c6d1", + "sourceSha": "f7c7f581088468e0c4074836603d2f90fcd5986b", + "hardware": "H200", + "execution": "complete", + "observedAt": "2026-10-17T05:07:27.401208Z", + "kind": "observation", + "observations": [ + { + "id": "a17baa4e13b3f1de1f70796e892d411a6efd2bcb5314b55d3e2c01d868a4c6d1:c1", + "cell": "c1", + "hardware": "H200", + "concurrency": 1, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 4 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11 · A single continuous medium shot of a drummer tapping a snare drum three times in", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"drum-taps\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":4,\"flow_shift\":12,\"fps\":24,\"frame_count\":107,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 4, + "completed": 4, + "scheduled": 4, + "failed": 0, + "samples": 4, + "p50": 117.06558148947079, + "p90": null, + "clipsGpuHour": 15.375335780450111, + "energyKj": 160.697757273, + "participating": 2, + "allocated": 2, + "replicas": null, + "wallSeconds": 468.2824559288565, + "durationSeconds": 4, + "frameCount": 107, + "avgPowerW": 1372.656654, + "enforcedLimitW": null, + "server": { + "tp": 1, + "ulysses": 2, + "attention": null + } + }, + { + "id": "a17baa4e13b3f1de1f70796e892d411a6efd2bcb5314b55d3e2c01d868a4c6d1:c2", + "cell": "c2", + "hardware": "H200", + "concurrency": 2, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 4 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11 · A single continuous medium shot of a drummer tapping a snare drum three times in", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"drum-taps\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":4,\"flow_shift\":12,\"fps\":24,\"frame_count\":107,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 4, + "completed": 4, + "scheduled": 4, + "failed": 0, + "samples": 4, + "p50": 233.34144142945297, + "p90": null, + "clipsGpuHour": 15.409529105389542, + "energyKj": 160.78275492999998, + "participating": 2, + "allocated": 2, + "replicas": null, + "wallSeconds": 467.2433499270119, + "durationSeconds": 4, + "frameCount": 107, + "avgPowerW": 1376.436968, + "enforcedLimitW": null, + "server": { + "tp": 1, + "ulysses": 2, + "attention": null + } + }, + { + "id": "a17baa4e13b3f1de1f70796e892d411a6efd2bcb5314b55d3e2c01d868a4c6d1:c4", + "cell": "c4", + "hardware": "H200", + "concurrency": 4, + "runtime": "71de97b264b04dcd514cf904003028aefe9775c8", + "workload": "1344 × 768 · 4 s · 24 fps · 50 steps · MiniMaxAI/MiniMax-H3 @ 42ed227ee7df · seed 11 · A single continuous medium shot of a drummer tapping a snare drum three times in", + "workloadKey": "{\"plan\":{\"cases\":[{\"case_id\":\"drum-taps\",\"prompt\":\"A single continuous medium shot of a drummer tapping a snare drum three times in a quiet rehearsal room. The drumsticks visibly contact the drumhead on each tap. The three crisp snare hits and the natural room ambience are audible. The camera remains still. No music, speech, subtitles, or scene cuts.\",\"requires_motion\":true,\"requires_sound\":true,\"seed\":11}],\"generation\":{\"aspect_ratio\":\"16:9\",\"audio_channels\":2,\"audio_flow_shift\":3,\"audio_sample_rate_hz\":32000,\"duration_seconds\":4,\"flow_shift\":12,\"fps\":24,\"frame_count\":107,\"height\":768,\"num_inference_steps\":50,\"width\":1344},\"model_id\":\"MiniMaxAI/MiniMax-H3\",\"model_revision\":\"42ed227ee7df40d41602854ae760620d6eb651fe\"},\"semantics\":{\"boundary\":\"submit_to_downloaded_media\",\"mode\":\"closed_loop\",\"server\":{\"performance_mode\":\"speed\"}}}", + "model": "MiniMaxAI/MiniMax-H3", + "status": "complete", + "valid": 4, + "completed": 4, + "scheduled": 4, + "failed": 0, + "samples": 4, + "p50": 292.02532795863226, + "p90": null, + "clipsGpuHour": 15.409334407778596, + "energyKj": 160.502909006, + "participating": 2, + "allocated": 2, + "replicas": null, + "wallSeconds": 467.24925356707536, + "durationSeconds": 4, + "frameCount": 107, + "avgPowerW": 1374.023888, + "enforcedLimitW": null, + "server": { + "tp": 1, + "ulysses": 2, + "attention": null + } + } + ], + "fidelity": null, + "calibration": null, + "releaseQualified": false, + "powerVerified": true, + "error": null + } + ], + "error": null + } + ], + "nextPage": null +} diff --git a/packages/app/package.json b/packages/app/package.json index fb2463cde..25b2fed13 100644 --- a/packages/app/package.json +++ b/packages/app/package.json @@ -31,6 +31,7 @@ "cache:invalidate": "bun --env-file=../../.env scripts/invalidate-cache.ts", "cache:warmup": "bun --env-file=../../.env scripts/warmup-cache.ts", "capture:fixtures": "bun scripts/capture-cypress-fixtures.ts", + "fixtures:video-history": "bun scripts/build-video-history-fixture.ts", "guard:zh-copy:chinese-only": "bun scripts/check-zh-chinese-only.ts --chinese-only" }, "dependencies": { diff --git a/packages/app/scripts/build-video-history-fixture.ts b/packages/app/scripts/build-video-history-fixture.ts new file mode 100644 index 000000000..e094cf935 --- /dev/null +++ b/packages/app/scripts/build-video-history-fixture.ts @@ -0,0 +1,82 @@ +/** + * Rebuild cypress/fixtures/api/video-history.json from retained StoredArtifact + * exports (frontend exports of the original CI artifacts; no new benchmark and + * no new publication). Entries whose run has no local export are kept verbatim, + * so fidelity-only entries survive a rebuild untouched. + * + * Usage: + * bun scripts/build-video-history-fixture.ts .json> + * + * Exports for runs the fixture does not list yet are added as new entries. + */ + +import { readdir, readFile, writeFile } from 'node:fs/promises'; +import { resolve } from 'node:path'; + +import { + videoHistoryEntry, + type VideoHistoryEntry, + type VideoHistoryPage, +} from '../src/components/video-benchmark/history'; +import type { StoredArtifact } from '../src/components/video-benchmark/stored'; +import { + FIXTURE_MANIFEST_FILENAME, + type FixtureManifest, + assertFixtureContent, + fixtureSha256, + fixtureTopLevel, +} from '../src/lib/testing/test-fixture-manifest'; + +const FIXTURE_NAME = 'video-history'; +const fixturesDir = resolve(import.meta.dirname, '..', 'cypress', 'fixtures', 'api'); +const source = process.argv[2]; +if (!source) throw new Error('Pass the directory holding retained .json exports'); + +const current = JSON.parse( + await readFile(resolve(fixturesDir, `${FIXTURE_NAME}.json`), 'utf8'), +) as VideoHistoryPage; +const exports = new Map(); +const files = await readdir(source); +for (const file of files.filter((name) => /^\d+\.json$/u.test(name))) { + const saved = JSON.parse(await readFile(resolve(source, file), 'utf8')) as StoredArtifact; + if (saved.storageVersion !== 1) throw new Error(`${file} is not a storageVersion 1 export`); + exports.set(`${saved.runId}.${saved.artifact.id}`, saved); +} + +let rebuilt = 0; +const known = new Set(current.entries.map((entry) => entry.id)); +const entries: VideoHistoryEntry[] = current.entries.map((entry) => { + const saved = exports.get(entry.id); + if (!saved) return entry; + rebuilt += 1; + const next = videoHistoryEntry(saved, entry.publishedAt); + // The existing fixture already carries the artifact identity (size, digest); keep it. + return { ...next, artifact: { ...next.artifact, ...entry.artifact } }; +}); +// An export the fixture has not seen becomes a new entry after the existing ones: an export +// is not a publication, so its publication time stays unknown and it sorts after dated entries. +const added = [...exports.values()].filter( + (saved) => !known.has(`${saved.runId}.${saved.artifact.id}`), +); +entries.push(...added.map((saved) => videoHistoryEntry(saved, null))); +const page: VideoHistoryPage = { schemaVersion: 1, entries, nextPage: current.nextPage }; +assertFixtureContent(FIXTURE_NAME, page); +const body = `${JSON.stringify(page, null, 2)}\n`; +await writeFile(resolve(fixturesDir, `${FIXTURE_NAME}.json`), body); + +const manifestPath = resolve(fixturesDir, FIXTURE_MANIFEST_FILENAME); +const manifest = JSON.parse(await readFile(manifestPath, 'utf8')) as FixtureManifest; +const capturedAt = new Date().toISOString(); +manifest.fixtures[FIXTURE_NAME] = { + bytes: Buffer.byteLength(body), + capturedAt, + sha256: fixtureSha256(body), + source: + 'Retained public H3 metadata replay from original runs via videoHistoryEntry (canonical workload identity, per-cell GPU counts, board power and server layout); publication time unavailable. No new benchmark or publication.', + topLevel: fixtureTopLevel(page), +}; +manifest.generatedAt = capturedAt; +await writeFile(manifestPath, `${JSON.stringify(manifest, null, 2)}\n`); +console.log( + `Wrote ${entries.length} entries; ${rebuilt} rebuilt and ${added.length} added from ${exports.size} exports`, +); diff --git a/packages/app/scripts/export-stored-artifact.ts b/packages/app/scripts/export-stored-artifact.ts new file mode 100644 index 000000000..05419516c --- /dev/null +++ b/packages/app/scripts/export-stored-artifact.ts @@ -0,0 +1,28 @@ +// Export a downloaded h3-results bundle directory as a storageVersion 1 StoredArtifact JSON, +// the frontend's retained form of a CI artifact. Usage: +// bun export-stored-artifact.ts +import { readFile, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { loadBundle } from '@/components/video-benchmark/bundle'; +import type { StoredArtifact } from '@/components/video-benchmark/stored'; + +const [dir, runId, artifactId, name, size, out] = process.argv.slice(2); +if (!dir || !runId || !artifactId || !name || !size || !out) + throw new Error('six arguments required'); +const bundle = await loadBundle(async (path) => new Blob([await readFile(join(dir, path))])); +const stored: StoredArtifact = { + storageVersion: 1, + runId, + artifact: { id: Number(artifactId), name, expired: false, size_in_bytes: Number(size) }, + sources: [ + { + id: runId, + documents: [...bundle.documents], + checksums: [...bundle.checksums], + assets: [], + texts: [], + }, + ], +}; +await writeFile(out, `${JSON.stringify(stored)}\n`); +console.log(`wrote ${out}: ${bundle.documents.size} documents, ${bundle.checksums.size} checksums`); diff --git a/packages/app/scripts/warm-video-results.ts b/packages/app/scripts/warm-video-results.ts index 9c4c54cb8..60a727082 100644 --- a/packages/app/scripts/warm-video-results.ts +++ b/packages/app/scripts/warm-video-results.ts @@ -1,4 +1,5 @@ import { env } from 'node:process'; +import { publishVideoResults } from '../src/lib/video/video-publication'; const productionOrigin = 'https://inferencex.semianalysis.com'; async function get(path: string) { const response = await fetch(`${productionOrigin}/api/video-runs${path}`, { @@ -9,18 +10,10 @@ async function get(path: string) { throw new Error('BLOB_READ_WRITE_TOKEN is not configured on the frontend'); return response.json(); } -const runId = env.H3_RUN_ID; -if (runId && !/^[1-9]\d*$/u.test(runId)) throw new Error('Invalid H3_RUN_ID'); -const catalog = runId ? { runs: [{ id: runId }] } : await get('?page=1'); -const runs: { id: number | string }[] = catalog.runs; -for (const run of runs) { - const data = await get(`?run=${run.id}`); - for (const artifact of data.artifacts) { - if (artifact.stored || artifact.expired) continue; - const result = await get(`?run=${run.id}&artifact=${artifact.id}&format=media`); - if (result.storageVersion !== 1) throw new Error('Unexpected H3 storage response'); - console.log( - `Stored H3 run ${run.id}, artifact ${artifact.id}, ${result.sources.length} executions`, - ); - } -} +await publishVideoResults({ + get, + runId: env.H3_RUN_ID, + onStored: (run, artifact, sources) => { + console.log(`Stored video run ${run}, artifact ${artifact}, ${sources} executions`); + }, +}); diff --git a/packages/app/src/app/(dashboard)/video/page.tsx b/packages/app/src/app/(dashboard)/video/page.tsx index 1a3c8f965..9b4d7cae5 100644 --- a/packages/app/src/app/(dashboard)/video/page.tsx +++ b/packages/app/src/app/(dashboard)/video/page.tsx @@ -1,5 +1,5 @@ import type { Metadata } from 'next'; -import VideoCIRuns from '@/components/video-benchmark/VideoCIRuns'; +import VideoDashboard from '@/components/video-benchmark/VideoDashboard'; import { tabMetadata } from '@/lib/routing/tab-meta'; export const metadata: Metadata = { @@ -7,5 +7,5 @@ export const metadata: Metadata = { robots: { index: false, follow: false }, }; export default function VideoPage() { - return ; + return ; } diff --git a/packages/app/src/app/api/v1/views/video/route.test.ts b/packages/app/src/app/api/v1/views/video/route.test.ts new file mode 100644 index 000000000..a20b39f8a --- /dev/null +++ b/packages/app/src/app/api/v1/views/video/route.test.ts @@ -0,0 +1,330 @@ +import { compareMetrics } from '@/components/video-benchmark/compare'; +import { resolveCompareSelection } from '@/components/video-benchmark/compare-url-state'; +import { concurrencyPlateau, powerUtilization } from '@/components/video-benchmark/evidence'; +import type { + VideoHistoryObservation, + VideoHistoryPage, +} from '@/components/video-benchmark/history'; +import { metricValue } from '@/components/video-benchmark/metrics'; +import { listedVideoCells, plotVideoPoints } from '@/components/video-benchmark/plot'; +import { dashboardCells, videoPoints } from '@/components/video-benchmark/points'; +import { servingFixture } from '@/components/video-benchmark/serving.fixture'; +import type { StoredArtifact } from '@/components/video-benchmark/stored'; +import { + metricOptions, + readVideoDashboardState, +} from '@/components/video-benchmark/video-url-state'; +import { NextRequest } from 'next/server'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { GET } from './route'; + +const source = vi.hoisted(() => vi.fn()); +vi.mock('@/app/api/video-runs/route', () => ({ GET: source })); + +const observation: VideoHistoryObservation = { + id: 'h200:c1', + cell: 'c1', + hardware: 'NVIDIA H200', + concurrency: 1, + runtime: 'frozen-runtime', + workload: '1344x768 · 8s · 24fps · 50 steps · MiniMax-H3', + workloadKey: 'frozen-workload', + model: 'MiniMaxAI/MiniMax-H3', + status: 'complete', + valid: 20, + completed: 20, + scheduled: 20, + failed: 0, + samples: 20, + p50: 100, + p90: 110, + clipsGpuHour: 5, + energyKj: 400, + participating: 4, + allocated: 8, + replicas: 1, + wallSeconds: 3600, + durationSeconds: 8, + frameCount: 192, + avgPowerW: 2000, + enforcedLimitW: 2800, + server: { tp: 2, ulysses: 2, attention: 'auto' }, +}; + +function page(observations: VideoHistoryObservation[], nextPage: number | null): VideoHistoryPage { + return { + schemaVersion: 1, + nextPage, + entries: [ + { + id: '123.456', + runId: '123', + publishedAt: '2026-09-20T00:00:00Z', + error: null, + artifact: { + id: 456, + name: 'h3-video-123-1', + expired: false, + stored: true, + size_in_bytes: 1, + }, + sources: [ + { + id: '123', + sha256: 'a'.repeat(64), + sourceSha: 'b'.repeat(40), + hardware: 'NVIDIA H200', + execution: 'complete', + observedAt: '2026-09-19T00:00:00Z', + kind: 'observation', + observations, + fidelity: null, + calibration: null, + releaseQualified: false, + error: null, + }, + ], + }, + ], + }; +} + +const pages = [ + page( + [ + observation, + { + ...observation, + id: 'h200:8g', + participating: 8, + p50: 130, + p90: 140, + server: { tp: 2, ulysses: 4, attention: 'auto' }, + }, + { ...observation, id: 'h200:c2', cell: 'c2', concurrency: 2, p50: 200, p90: 210 }, + { + ...observation, + id: 'smoke:c1', + hardware: 'NVIDIA B200', + samples: 3, + workload: 'short smoke', + workloadKey: 'short-smoke', + p90: null, + }, + ], + 2, + ), + page( + [ + { + ...observation, + id: 'h100:c1', + hardware: 'NVIDIA H100 80GB HBM3', + p50: 120, + p90: 130, + wallSeconds: 4000, + }, + { ...observation, p50: 999, p90: 999 }, // older re-export must not replace the first + ], + null, + ), +]; +const request = (query = '') => new NextRequest(`https://example.test/api/v1/views/video?${query}`); + +beforeEach(() => { + source.mockReset(); + source.mockImplementation((req: NextRequest) => { + if (req.nextUrl.searchParams.get('format') === 'history') + return Response.json(pages[Number(req.nextUrl.searchParams.get('page')) - 1]); + return new Response(null, { status: 204 }); + }); +}); + +describe('published VideoGenX dashboard projection', () => { + it('defaults to the current dashboard, reads history pages and preserves numerical parity', async () => { + const response = await GET(request()); + expect(response.status).toBe(200); + expect(response.headers.get('cache-control')).toContain('no-store'); + const body = await response.json(); + const state = readVideoDashboardState(''); + const { cells } = dashboardCells(videoPoints(pages)); + const options = metricOptions(state); + expect(body.params.view).toBe('dashboard'); + expect(body.coverage).toEqual({ pagesRead: 2, maxPages: 5, nextPage: null, truncated: false }); + expect(body.plot).toEqual(plotVideoPoints(cells, state, () => '', new Set())); + expect(body.rows.map((row: { point: { id: string } }) => row.point.id)).toEqual( + listedVideoCells(cells, state, new Set()).map((p) => p.id), + ); + expect( + body.rows.find((row: { point: { id: string } }) => row.point.id === 'h200:c1').metrics, + ).toMatchObject({ + p50Latency: 100, + p90Latency: 110, + videosPerGpuHour: 5, + dollarsPerVideo: metricValue( + cells.find((p) => p.id === 'h200:c1')!, + 'dollarsPerVideo', + options, + ), + }); + expect(body.evidence.power).toEqual(powerUtilization(cells)); + expect(body.evidence.plateau).toEqual(concurrencyPlateau(cells)); + expect(body.comparison.cases.status).toBe('not-requested'); + expect(source).toHaveBeenCalledTimes(2); + expect( + source.mock.calls.every(([req]) => req.nextUrl.searchParams.get('format') === 'history'), + ).toBe(true); + }); + + it('shares axis, tier, optimal, API price, compare and hardware-filter semantics with the UI', async () => { + const query = + 'v_x=p50Latency&v_y=dollarsPerVideo&v_tier=r&v_optimal=0&v_api=0.08&v_hidden=h100&v_base=h200&v_cand=h100&v_case=3'; + const response = await GET(request(query)); + expect(response.status).toBe(200); + const body = await response.json(); + const state = readVideoDashboardState(query); + const { cells } = dashboardCells(videoPoints(pages)); + const options = metricOptions(state); + expect(body.plot).toEqual(plotVideoPoints(cells, state, () => '', new Set(['h100']))); + expect(body.rows.map((row: { point: { id: string } }) => row.point.id)).toEqual([ + 'h200:c1', + 'h200:8g', + ]); + expect( + body.rows.every( + (row: { metrics: { apiPricePerVideo: number } }) => row.metrics.apiPricePerVideo === 0.64, + ), + ).toBe(true); + const pair = resolveCompareSelection( + { baseline: 'h200', candidate: 'h100', caseIndex: 3 }, + cells, + ); + expect(body.comparison.metrics).toEqual( + compareMetrics(pair.baseline!, pair.candidate!, options), + ); + expect(body.params).toMatchObject({ + hidden: ['h100'], + baseline: 'h200', + candidate: 'h100', + caseIndex: 3, + }); + expect( + body.kpis.find((row: { hardwareKey: string }) => row.hardwareKey === 'h100').point, + ).not.toBeNull(); + expect(body.evidence.power).toEqual(powerUtilization(cells)); + }); + + it('reports the same five-page bound and never silently presents truncated history as complete', async () => { + source.mockImplementation((req: NextRequest) => + Response.json(page([], Number(req.nextUrl.searchParams.get('page')) + 1)), + ); + const response = await GET(request()); + const body = await response.json(); + expect(source).toHaveBeenCalledTimes(5); + expect(body.coverage).toEqual({ pagesRead: 5, maxPages: 5, nextPage: 6, truncated: true }); + expect(body.rows).toEqual([]); + }); + + it('keeps missing published comparison media distinct from an empty successful matched case list', async () => { + const response = await GET(request('view=compare&v_base=h200&v_cand=h100&v_case=8')); + expect(response.status).toBe(200); + const body = await response.json(); + expect(body.comparison.cases).toMatchObject({ + status: 'not-published', + selected: null, + resolvedCaseIndex: 0, + }); + expect( + source.mock.calls.filter(([req]) => req.nextUrl.searchParams.get('format') === 'published'), + ).toHaveLength(1); + }); + + it('pairs published cases, clamps the requested index and excludes asset URLs and raw documents', async () => { + const fixture = servingFixture(); + fixture.documents.set('manifest.json', fixture.manifest); + fixture.documents.set('ci.json', fixture.ci); + fixture.checksums.set('manifest.json', 'a'.repeat(64)); + const saved: StoredArtifact = { + storageVersion: 1, + runId: '123', + artifact: { id: 456, name: 'published', expired: false, size_in_bytes: 1 }, + sources: [ + { + id: '123', + documents: [...fixture.documents], + checksums: [...fixture.checksums], + texts: [], + assets: [ + [ + 'gpu/c1/baseline/artifacts/measurement-r001-c001.mp4', + { + url: 'https://private.test/secret-token', + downloadUrl: 'https://private.test/download', + }, + ], + ], + }, + ], + }; + source.mockImplementation((req: NextRequest) => + Response.json( + req.nextUrl.searchParams.get('format') === 'published' + ? saved + : pages[Number(req.nextUrl.searchParams.get('page')) - 1], + ), + ); + const response = await GET(request('view=compare&v_case=9999')); + const body = await response.json(); + expect(body.comparison.cases).toMatchObject({ + status: 'ready', + count: 4, + resolvedCaseIndex: 3, + unmatched: { baseline: 0, candidate: 0 }, + selected: { + repetition: 3, + baseline: { slotId: 'measurement-r004-c001', seconds: 120 }, + candidate: { slotId: 'measurement-r004-c001', seconds: 120 }, + }, + }); + expect(JSON.stringify(body)).not.toMatch(/private\.test|secret-token|\/runtime\/sglang/); + expect(body.comparison.cases.selected.baseline.mediaPath).toBe( + 'gpu/c1/baseline/artifacts/measurement-r004-c001.mp4', + ); + }); + + it('retains explicit discovery and treats media read failure as unavailable, not zero pairs', async () => { + source.mockImplementation(() => Response.json({ runs: [] })); + const discoveryResponse = await GET(request('view=discovery&page=2')); + const discovery = await discoveryResponse.json(); + expect(discovery.discovery).toEqual({ runs: [] }); + expect(source.mock.calls[0][0].nextUrl.searchParams.get('page')).toBe('2'); + source.mockImplementation((req: NextRequest) => + req.nextUrl.searchParams.get('format') === 'published' + ? Response.json({ error: 'secret-token' }, { status: 503 }) + : Response.json(pages[Number(req.nextUrl.searchParams.get('page')) - 1]), + ); + const compareResponse = await GET(request('view=compare')); + const body = await compareResponse.json(); + expect(body.comparison.cases.status).toBe('unavailable'); + expect(body.rows.length).toBeGreaterThan(0); + expect(JSON.stringify(body)).not.toContain('secret-token'); + }); + + it('rejects mixed modes and strips source errors instead of leaking private details', async () => { + for (const query of [ + 'view=dashboard&run=123', + 'run=123&v_x=p50Latency', + 'page=2', + 'v_hidden=h100&v_hidden=h200', + ]) { + const response = await GET(request(query)); + expect(response.status).toBe(400); + } + source.mockImplementation(() => + Response.json({ error: 'secret-token https://private.test' }, { status: 503 }), + ); + const response = await GET(request()); + expect(response.status).toBe(503); + expect(await response.json()).toEqual({ error: 'Source data unavailable' }); + }); +}); diff --git a/packages/app/src/app/api/v1/views/video/route.ts b/packages/app/src/app/api/v1/views/video/route.ts index c83eecec4..631816c43 100644 --- a/packages/app/src/app/api/v1/views/video/route.ts +++ b/packages/app/src/app/api/v1/views/video/route.ts @@ -1,4 +1,9 @@ import { GET as videoRuns } from '@/app/api/video-runs/route'; +import { compareSide, pairCases } from '@/components/video-benchmark/compare'; +import { + VIDEO_HISTORY_MAX_PAGES, + type VideoHistoryPage, +} from '@/components/video-benchmark/history'; import { servingCells } from '@/components/video-benchmark/serving'; import { storedBundle, type StoredArtifact } from '@/components/video-benchmark/stored'; import { @@ -18,11 +23,102 @@ import { } from '@/lib/views-api/params'; import { VIEW_QUERY_PARAMS } from '@/lib/views-api/registry'; import { readResponse, sourceRequest } from '@/lib/views-api/source'; +import { videoDashboardProjection } from '@/lib/views-api/video-dashboard'; import { type NextRequest, NextResponse } from 'next/server'; export const dynamic = 'force-dynamic'; export const maxDuration = 300; const headers = { 'Cache-Control': 'private, no-store' }; + +async function dashboardResponse(request: NextRequest, view: 'dashboard' | 'compare') { + const pages: VideoHistoryPage[] = []; + let nextPage: number | null = 1; + for (let page = 1; page <= VIDEO_HISTORY_MAX_PAGES; page++) { + const payload = await readResponse( + await videoRuns( + sourceRequest(request, '/api/video-runs', { format: 'history', page: String(page) }), + ), + ); + if (payload.schemaVersion !== 1 || !Array.isArray(payload.entries)) + throw new Error('Unsupported published history projection'); + pages.push(payload); + nextPage = payload.nextPage; + if (nextPage === null) break; + } + const projection = videoDashboardProjection(pages, request.nextUrl.search); + const { baseline, candidate } = projection.comparison; + let cases: { + status: 'not-requested' | 'no-pair' | 'not-published' | 'ready' | 'unavailable'; + resolvedCaseIndex: number; + count: number; + selected: ReturnType['pairs'][number] | null; + unmatched: ReturnType['unmatched'] | null; + } = { status: 'not-requested', resolvedCaseIndex: 0, count: 0, selected: null, unmatched: null }; + if (view === 'compare') { + cases.status = 'no-pair'; + if (baseline && candidate) { + // Same artifact can contain both points. Read it once, through the published-only media path. + const artifacts = new Map(); + try { + for (const point of [baseline, candidate]) { + const key = `${point.runId}:${point.artifactId}`; + if (artifacts.has(key)) continue; + const response = await videoRuns( + sourceRequest(request, '/api/video-runs', { + run: point.runId, + artifact: String(point.artifactId), + format: 'published', + }), + ); + artifacts.set( + key, + response.status === 204 ? null : await readResponse(response), + ); + } + const b = artifacts.get(`${baseline.runId}:${baseline.artifactId}`); + const c = artifacts.get(`${candidate.runId}:${candidate.artifactId}`); + cases.status = 'not-published'; + if (b && c) { + const matched = pairCases( + compareSide(b, baseline).records, + compareSide(c, candidate).records, + ); + const index = Math.min( + projection.params.caseIndex, + Math.max(matched.pairs.length - 1, 0), + ); + cases = { + status: 'ready', + resolvedCaseIndex: index, + count: matched.pairs.length, + selected: matched.pairs[index] ?? null, + unmatched: matched.unmatched, + }; + } + } catch { + // Like the collapsed UI panel, a media error does not erase published chart metrics. + cases.status = 'unavailable'; + } + } + } + return NextResponse.json( + { + view: 'video', + apiVersion: 'v1', + ...projection, + params: { ...projection.params, view, displayView: projection.params.view }, + coverage: { + pagesRead: pages.length, + maxPages: VIDEO_HISTORY_MAX_PAGES, + nextPage, + truncated: nextPage !== null, + }, + comparison: { ...projection.comparison, cases }, + }, + { headers }, + ); +} + function serializePoint({ run: evidenceRun, ...point @@ -42,6 +138,24 @@ export function GET(request: NextRequest) { return runViewsRoute('video', async () => { validateViewParams(request.nextUrl.searchParams, VIEW_QUERY_PARAMS['video']); const s = request.nextUrl.searchParams; + const mode = parseEnumParam( + s.get('view'), + 'view', + ['dashboard', 'compare', 'discovery', 'results', 'tradeoff'], + s.has('run') || s.has('artifact') ? 'results' : 'dashboard', + ); + if (mode === 'dashboard' || mode === 'compare') { + for (const key of s.keys()) + if (key !== 'view' && !key.startsWith('v_')) + throw new ViewsApiParamError(key, 'Use a legacy view for run/artifact selectors'); + return dashboardResponse(request, mode); + } + for (const key of s.keys()) + if (key.startsWith('v_')) + throw new ViewsApiParamError( + key, + 'Dashboard selectors require view=dashboard or view=compare', + ); // GitHub run id and artifact id share the positive-digits rule. const run = parseRunIdParam(s.get('run'), 'run'); const artifact = parseRunIdParam(s.get('artifact'), 'artifact'); @@ -176,7 +290,7 @@ export function GET(request: NextRequest) { ['participating', 'allocated'], 'participating', ); - const view = parseEnumParam(s.get('view'), 'view', ['results', 'tradeoff'], 'results'); + const view = mode; const cell = s.get('cell'), slot = s.get('slot'); const evidence = selectedSource diff --git a/packages/app/src/app/api/video-runs/route.test.ts b/packages/app/src/app/api/video-runs/route.test.ts index ebaa13fc1..faca20c80 100644 --- a/packages/app/src/app/api/video-runs/route.test.ts +++ b/packages/app/src/app/api/video-runs/route.test.ts @@ -2,6 +2,7 @@ import { NextRequest } from 'next/server'; import { afterEach, describe, expect, it, vi } from 'vitest'; import { GET } from './route'; import { + publishedVideoHistory, readStoredArtifact, storedArtifacts, storeVideoArtifact, @@ -10,6 +11,9 @@ import { vi.mock('@/lib/video/video-storage', () => ({ videoStorageEnabled: vi.fn(() => false), + publishedVideoHistory: vi.fn(() => + Promise.resolve({ schemaVersion: 1, entries: [], nextPage: null }), + ), storedArtifacts: vi.fn(() => Promise.resolve([])), readStoredArtifact: vi.fn(), storeVideoArtifact: vi.fn(), @@ -22,6 +26,7 @@ afterEach(() => { fetchMock.mockReset(); vi.mocked(videoStorageEnabled).mockReturnValue(false); vi.mocked(storedArtifacts).mockResolvedValue([]); + vi.mocked(publishedVideoHistory).mockClear(); vi.mocked(readStoredArtifact).mockReset(); vi.mocked(storeVideoArtifact).mockClear(); vi.unstubAllEnvs(); @@ -29,6 +34,15 @@ afterEach(() => { }); describe('H3 CI artifact access', () => { + it('discovers old published results without scanning unrelated recent Actions or publishing', async () => { + fetchMock.mockResolvedValueOnce(response({ private: false })); + const result = await GET(request('?format=history&page=2')); + expect(await result.json()).toEqual({ schemaVersion: 1, entries: [], nextPage: null }); + expect(publishedVideoHistory).toHaveBeenCalledWith(2); + expect(fetchMock).toHaveBeenCalledTimes(1); + expect(storeVideoArtifact).not.toHaveBeenCalled(); + }); + it('discovers benchmark runs without including branch-named unit jobs', async () => { fetchMock.mockResolvedValueOnce(response({ private: false })).mockResolvedValueOnce( response({ @@ -37,12 +51,13 @@ describe('H3 CI artifact access', () => { { id: 2, name: 'Test H3 Video', path: '.github/workflows/test-h3-video.yml' }, { id: 3, name: 'H3 Video Smoke', path: '.github/workflows/h3-video.yml' }, { id: 4, name: 'H3 retained-media fidelity', path: '.github/workflows/h3-fidelity.yml' }, + { id: 5, name: 'InferenceX Sweep', path: '.github/workflows/run-sweep.yml' }, ], }), ); const result = await GET(request()); const data = await result.json(); - expect(data.runs.map((r: { id: number }) => r.id)).toEqual([1, 3, 4]); + expect(data.runs.map((r: { id: number }) => r.id)).toEqual([1, 3, 4, 5]); expect(result.headers.get('cache-control')).toContain('no-store'); }); it('refuses a private repository before reading any artifacts', async () => { @@ -68,33 +83,37 @@ describe('H3 CI artifact access', () => { const result3 = await GET(request('?run=10&artifact=20')); expect(result3.status).toBe(410); }); - it('streams the verified run artifact with a deadline longer than metadata requests', async () => { - vi.stubEnv('NODE_ENV', 'development'); - const deadline = new AbortController().signal; - const timeout = vi - .spyOn(AbortSignal, 'timeout') - .mockImplementation((ms) => (ms === 270000 ? deadline : new AbortController().signal)); - fetchMock - .mockResolvedValueOnce(response({ private: false })) - .mockResolvedValueOnce( - response({ - name: 'h3-results-10-1', - workflow_run: { id: 10 }, - size_in_bytes: 4, - archive_download_url: 'https://evil.example', - }), - ) - .mockResolvedValueOnce(new Response(new Uint8Array([80, 75, 3, 4]))); - const result = await GET(request('?run=10&artifact=20')); - expect(timeout).toHaveBeenCalledWith(270000); - expect(fetchMock.mock.calls[2][1]?.signal).toBe(deadline); - expect(fetchMock.mock.calls[0][1]?.signal).not.toBe(deadline); - expect(result.headers.get('content-type')).toBe('application/zip'); - expect([...new Uint8Array(await result.arrayBuffer())]).toEqual([80, 75, 3, 4]); - expect(String(fetchMock.mock.calls[2][0])).toBe( - 'https://api.github.com/repos/SemiAnalysisAI/InferenceX/actions/artifacts/20/zip', - ); - }); + it.each(['h3-results-10-1', 'video-serving-10-1', 'h3-results-10-1-h3-sglang-h200-bf16'])( + 'streams %s with a deadline longer than metadata requests', + async (name) => { + vi.stubEnv('NODE_ENV', 'development'); + const deadline = new AbortController().signal; + const timeout = vi + .spyOn(AbortSignal, 'timeout') + .mockImplementation((ms) => (ms === 270000 ? deadline : new AbortController().signal)); + fetchMock + .mockResolvedValueOnce(response({ private: false })) + .mockResolvedValueOnce( + response({ + name, + workflow_run: { id: 10 }, + size_in_bytes: 4, + archive_download_url: 'https://evil.example', + }), + ) + .mockResolvedValueOnce(new Response(new Uint8Array([80, 75, 3, 4]))); + const result = await GET(request('?run=10&artifact=20')); + expect(result.status).toBe(200); + expect(timeout).toHaveBeenCalledWith(270000); + expect(fetchMock.mock.calls[2][1]?.signal).toBe(deadline); + expect(fetchMock.mock.calls[0][1]?.signal).not.toBe(deadline); + expect(result.headers.get('content-type')).toBe('application/zip'); + expect([...new Uint8Array(await result.arrayBuffer())]).toEqual([80, 75, 3, 4]); + expect(String(fetchMock.mock.calls[2][0])).toBe( + 'https://api.github.com/repos/SemiAnalysisAI/InferenceX/actions/artifacts/20/zip', + ); + }, + ); it('rejects oversized artifacts before downloading', async () => { fetchMock.mockResolvedValueOnce(response({ private: false })).mockResolvedValueOnce( response({ @@ -118,6 +137,25 @@ describe('H3 CI artifact access', () => { artifacts: [], }); }); + it('discovers native row artifacts beyond the first run page without duplicating overlapping IDs', async () => { + const native = { id: 120, name: `h3-results-10-2-${'a'.repeat(64)}` }; + fetchMock + .mockResolvedValueOnce(response({ private: false })) + .mockResolvedValueOnce(response({ id: 10, conclusion: 'failure' })) + .mockResolvedValueOnce( + response({ + artifacts: Array.from({ length: 100 }, (_, id) => ({ id, name: 'unrelated' })), + }), + ) + .mockResolvedValueOnce( + response({ artifacts: [native, native, { id: 121, name: 'h3-results-11-1' }] }), + ); + const result = await GET(request('?run=10')); + const data = await result.json(); + expect(data.artifacts).toEqual([native]); + expect(String(fetchMock.mock.calls[3][0])).toContain('artifacts?per_page=100&page=2'); + expect(storeVideoArtifact).not.toHaveBeenCalled(); + }); it('falls back to the CI ZIP when object storage is not configured', async () => { fetchMock.mockResolvedValueOnce(response({ private: false })); const result = await GET(request('?run=10&artifact=20&format=media')); diff --git a/packages/app/src/app/api/video-runs/route.ts b/packages/app/src/app/api/video-runs/route.ts index ba187eddc..19463e429 100644 --- a/packages/app/src/app/api/video-runs/route.ts +++ b/packages/app/src/app/api/video-runs/route.ts @@ -1,3 +1,5 @@ +import { videoArtifactIdentity } from '@/components/video-benchmark/artifact-identity'; +import type { CIArtifact } from '@/components/video-benchmark/archive'; import { type NextRequest, NextResponse } from 'next/server'; import { GITHUB_API_BASE, @@ -6,7 +8,11 @@ import { SITE_URL, } from '@semianalysisai/inferencex-constants'; import { getGithubToken } from '@/lib/github/github-artifacts'; +import { loadFixture } from '@/lib/testing/test-fixtures'; +import { readVideoReplay } from '@/lib/video-replay'; +import type { VideoHistoryPage } from '@/components/video-benchmark/history'; import { + publishedVideoHistory, readStoredArtifact, storedArtifacts, storeVideoArtifact, @@ -19,7 +25,6 @@ const ROOT = `${GITHUB_API_BASE}/repos/${GITHUB_OWNER}/${GITHUB_REPO}`; const MAX_BYTES = 256 * 1024 ** 2; const headers = { 'Cache-Control': 'private, no-store' }; const id = (value: string) => /^[1-9]\d{0,19}$/u.test(value); -const artifactName = /^h3-(?:results|video|fidelity)-(?\d+)-(?\d+)$/u; function github(path: string, signal = AbortSignal.timeout(30000)) { const token = getGithubToken(); @@ -38,15 +43,31 @@ export async function GET(request: NextRequest) { const runId = query.get('run'); const artifactId = query.get('artifact'); const page = query.get('page') ?? '1'; + const history = query.get('format') === 'history'; const publishedOnly = query.get('format') === 'published'; const media = query.get('format') === 'media' || publishedOnly; - if ((runId && !id(runId)) || (artifactId && (!runId || !id(artifactId))) || !id(page)) + if ( + (runId && !id(runId)) || + (artifactId && (!runId || !id(artifactId))) || + !id(page) || + !Number.isSafeInteger(Number(page)) || + (history && (runId || artifactId)) + ) return NextResponse.json( { error: 'Invalid CI run, artifact or page' }, { status: 400, headers }, ); + if (history && process.env.E2E_FIXTURES === '1') { + const fixture = loadFixture('video-history'); + return NextResponse.json( + Number(page) === 1 ? fixture : { schemaVersion: 1, entries: [], nextPage: null }, + { headers: { ...headers, 'X-VideoGenX-Replay': '1' } }, + ); + } const deadline = AbortSignal.timeout(270000); try { + const replay = await readVideoReplay(query); + if (replay) return replay; if ( process.env.NODE_ENV === 'development' && !videoStorageEnabled() && @@ -54,6 +75,7 @@ export async function GET(request: NextRequest) { ) { // Production checks repository visibility; local reads must never publish on a cache miss. const url = new URL('/api/video-runs', SITE_URL); + if (history) url.searchParams.set('format', 'history'); if (runId) url.searchParams.set('run', runId); if (artifactId) { url.searchParams.set('artifact', artifactId); @@ -82,9 +104,11 @@ export async function GET(request: NextRequest) { ) throw new Error('Published artifact identity mismatch'); } else if ( - runId - ? String(data.run?.id) !== runId || !Array.isArray(data.artifacts) - : !Array.isArray(data.runs) + history + ? data.schemaVersion !== 1 || !Array.isArray(data.entries) + : runId + ? String(data.run?.id) !== runId || !Array.isArray(data.artifacts) + : !Array.isArray(data.runs) ) { throw new Error('Published run identity mismatch'); } @@ -99,6 +123,7 @@ export async function GET(request: NextRequest) { { error: 'Public H3 repository unavailable' }, { status: 503, headers }, ); + if (history) return NextResponse.json(await publishedVideoHistory(Number(page)), { headers }); if (artifactId) { if (media && !videoStorageEnabled()) return new Response(null, { status: 204, headers }); if (media) { @@ -117,8 +142,8 @@ export async function GET(request: NextRequest) { { status: metaResponse.status, headers }, ); const artifact = await metaResponse.json(); - const name = artifactName.exec(artifact.name ?? ''); - if (!name || name.groups?.runId !== runId || String(artifact.workflow_run?.id) !== runId) + const name = videoArtifactIdentity(artifact.name ?? ''); + if (!name || name.run !== runId || String(artifact.workflow_run?.id) !== runId) return NextResponse.json( { error: 'Artifact does not belong to this H3 run' }, { status: 404, headers }, @@ -161,7 +186,7 @@ export async function GET(request: NextRequest) { return new Response(stream, { headers: { ...headers, 'Content-Type': 'application/zip' } }); } if (runId) { - const [runResponse, artifactsResponse, saved] = await Promise.all([ + const [runResponse, firstArtifactsResponse, saved] = await Promise.all([ github(`/actions/runs/${runId}`), github(`/actions/runs/${runId}/artifacts?per_page=100`), storedArtifacts(runId).catch(() => []), @@ -171,13 +196,24 @@ export async function GET(request: NextRequest) { { error: 'CI run unavailable' }, { status: runResponse.status, headers }, ); - if (!artifactsResponse.ok) - return NextResponse.json({ error: 'Cannot list CI artifacts' }, { status: 502, headers }); - const data = await artifactsResponse.json(); - const artifacts = (data.artifacts ?? []).filter((a: { name: string }) => { - const match = artifactName.exec(a.name); - return match?.groups?.runId === runId; - }); + const artifacts: CIArtifact[] = []; + const seen = new Set(); + let artifactsResponse = firstArtifactsResponse; + for (let artifactPage = 1; ; artifactPage++) { + if (!artifactsResponse.ok) + return NextResponse.json({ error: 'Cannot list CI artifacts' }, { status: 502, headers }); + const data: { artifacts: CIArtifact[] } = await artifactsResponse.json(); + for (const artifact of data.artifacts ?? []) { + if (videoArtifactIdentity(artifact.name)?.run !== runId || seen.has(artifact.id)) + continue; + seen.add(artifact.id); + artifacts.push(artifact); + } + if ((data.artifacts?.length ?? 0) < 100) break; + artifactsResponse = await github( + `/actions/runs/${runId}/artifacts?per_page=100&page=${artifactPage + 1}`, + ); + } for (const stored of saved) { const index = artifacts.findIndex((a: { id: number }) => a.id === stored.id); if (index === -1) artifacts.push(stored); @@ -198,6 +234,7 @@ export async function GET(request: NextRequest) { runs: runs.filter( (run: { name: string; display_title: string; path: string }) => (run.path === '.github/workflows/e2e-tests.yml' && /\bh3\b/iu.test(run.name)) || + run.path === '.github/workflows/run-sweep.yml' || run.path === '.github/workflows/h3-video.yml' || run.path === '.github/workflows/h3-fidelity.yml', ), @@ -207,7 +244,11 @@ export async function GET(request: NextRequest) { ); } catch { return NextResponse.json( - { error: 'GitHub is unavailable; retry shortly' }, + { + error: history + ? 'Published video history is unavailable; retry shortly' + : 'GitHub is unavailable; retry shortly', + }, { status: 502, headers }, ); } diff --git a/packages/app/src/app/zh/(dashboard)/video/page.tsx b/packages/app/src/app/zh/(dashboard)/video/page.tsx index 885cbc279..ad8f5fafb 100644 --- a/packages/app/src/app/zh/(dashboard)/video/page.tsx +++ b/packages/app/src/app/zh/(dashboard)/video/page.tsx @@ -1,5 +1,5 @@ import type { Metadata } from 'next'; -import VideoCIRuns from '@/components/video-benchmark/VideoCIRuns'; +import VideoDashboard from '@/components/video-benchmark/VideoDashboard'; import { tabMetadataZh } from '@/lib/routing/tab-meta-zh'; export const metadata: Metadata = { @@ -7,5 +7,5 @@ export const metadata: Metadata = { robots: { index: false, follow: false }, }; export default function VideoPage() { - return ; + return ; } diff --git a/packages/app/src/components/video-benchmark/FidelityResults.tsx b/packages/app/src/components/video-benchmark/FidelityResults.tsx deleted file mode 100644 index 1c8042a39..000000000 --- a/packages/app/src/components/video-benchmark/FidelityResults.tsx +++ /dev/null @@ -1,493 +0,0 @@ -'use client'; - -import { useEffect, useState } from 'react'; -import { Card } from '@/components/ui/card'; -import { Heading } from '@/components/ui/heading'; -import { useLocale } from '@/lib/i18n/use-locale'; -import { track } from '@/lib/analytics/analytics'; -import VideoSelect from './VideoSelect'; -import { at, ROLES, rows, text, type Json } from './bundle'; -import { fidelitySource, loadFidelityBundle, type FidelityBundle } from './fidelity'; -import { storedFidelityBundle, type StoredSource } from './stored'; -import { renderReportHtml } from './report'; - -const STRINGS = { - en: { - title: 'Paired video fidelity', - intro: 'CPU comparison of retained C1 videos. No new video generation.', - pairs: 'Matched valid pairs', - outside: 'Pairs outside fidelity thresholds', - inconclusive: 'Inconclusive pairs', - calibration: 'Threshold calibration', - qualification: 'Release qualified', - unavailable: 'Unavailable', - uncalibrated: 'Uncalibrated', - yes: 'Yes', - no: 'No', - note: 'Technical validity covers decoding, geometry and continuity. Paired pixel/audio comparisons measure output drift using exploratory thresholds; they do not establish perceptual quality, prompt adherence or a hardware quality winner.', - loading: 'Loading and verifying paired evidence…', - error: 'Could not load paired evidence', - slot: 'Paired clip', - baseline: 'Baseline', - candidate: 'Candidate', - prompt: 'Prompt', - seed: 'Seed', - audio: 'Play one side at a time to compare original audio.', - download: 'Download original MP4', - noMedia: 'No media for this observation', - mediaError: 'The browser could not play this media. Download the original MP4 to inspect it.', - latency: 'Submission to downloaded media (s)', - integrity: 'Video/audio technical validity', - details: 'Generation settings and revisions', - runtime: 'Runtime revision', - model: 'Model revision', - metrics: 'Measured fidelity', - psnr: 'Video PSNR (dB)', - mae: 'Video MAE (normalized 0–1)', - identical: 'Identical frames; finite PSNR undefined', - spectral: 'Audio spectral cosine', - rms: 'Audio RMS ratio (candidate / baseline)', - waveform: 'Audio waveform MAE (PCM amplitude)', - frames: 'Compared / total video frames', - videoCoverage: 'Video sample coverage (fraction)', - audioCoverage: 'Audio sample coverage (fraction)', - audioSamples: 'Compared audio samples / channel', - checks: 'Recorded checks and thresholds', - check: 'Check', - status: 'Recorded status', - observed: 'Observed', - threshold: 'Threshold', - unit: 'Unit', - timing: - 'Delivery latency is descriptive and includes polling and download. These cross-hardware results are not a calibrated performance regression or serving-capacity claim.', - sources: 'Original GPU runs and evidence', - originalResult: 'Original serving result and runtime log', - runtimeNote: - 'Resolved attention backends and runtime settings can differ across sources. Consult the original runtime logs before attributing output differences to hardware.', - producer: 'CPU comparison CI', - artifact: 'Download CI artifact', - digest: 'Comparison SHA256', - raw: 'Raw comparison JSON', - report: 'Original report', - limitations: 'Backend policy and limitations', - }, - zh: { - title: '成对视频保真度', - intro: '使用保留的 C1 视频在 CPU 上进行对比,未重新生成视频。', - pairs: '匹配的有效视频对', - outside: '超出保真度阈值的视频对', - inconclusive: '无法判定的视频对', - calibration: '阈值校准状态', - qualification: '是否通过发布验收', - unavailable: '无数据', - uncalibrated: '未校准', - yes: '是', - no: '否', - note: '技术有效性涵盖解码、画面尺寸与连续性。像素与音频配对比较采用探索性阈值,衡量的是输出偏差,不能证明感知质量或 prompt 遵循程度,也不能据此判定哪种硬件的视频质量更好。', - loading: '正在加载并校验配对证据…', - error: '无法加载配对证据', - slot: '配对视频', - baseline: '基线', - candidate: '候选', - prompt: 'Prompt', - seed: 'Seed', - audio: '每次只播放一侧,以便对比原始音频。', - download: '下载原始 MP4', - noMedia: '此侧暂无可用媒体', - mediaError: '浏览器无法播放此媒体,请下载原始 MP4 检查。', - latency: '提交到视频下载完成(秒)', - integrity: '视频与音频技术有效性', - details: '生成设置与版本', - runtime: '运行时版本', - model: '模型版本', - metrics: '实测保真度', - psnr: '视频 PSNR(dB)', - mae: '视频 MAE(归一化 0–1)', - identical: '帧完全一致,PSNR 无有限值', - spectral: '音频频谱余弦相似度', - rms: '音频 RMS 比值(候选 / 基线)', - waveform: '音频波形 MAE(PCM 幅度)', - frames: '已比较 / 总视频帧数', - videoCoverage: '视频采样覆盖比例', - audioCoverage: '音频采样覆盖比例', - audioSamples: '每声道已比较音频采样数', - checks: '已记录的检查与阈值', - check: '检查项', - status: '已记录状态', - observed: '观测值', - threshold: '阈值', - unit: '单位', - timing: - '交付延迟仅作描述,包含轮询与下载时间。这些跨硬件数据不构成已校准的性能回归或持续服务容量结论。', - sources: '原始 GPU 运行与证据', - originalResult: '原始服务测试结果与运行时日志', - runtimeNote: - '各来源实际使用的 attention backend 与运行时设置可能不同。判断输出差异的原因时,请核对原始运行时日志,不要直接归因于硬件。', - producer: 'CPU 对比 CI', - artifact: '下载 CI 产物', - digest: '对比结果 SHA256', - raw: '原始对比 JSON', - report: '原始报告', - limitations: '后端策略与局限', - }, -}; - -export default function FidelityResults({ - reader, - published, - runId, -}: { - reader?: (path: string) => Promise; - published?: StoredSource; - runId: string; -}) { - const locale = useLocale(); - const s = STRINGS[locale]; - const [loaded, setLoaded] = useState<{ - bundle: FidelityBundle; - urls: Map; - downloads: Map; - html: string; - } | null>(null); - const [loadError, setError] = useState(''); - const [slotId, setSlotId] = useState(''); - const [mediaError, setMediaError] = useState(false); - const [reportOpen, setReportOpen] = useState(false); - useEffect(() => { - let cancelled = false; - const created: string[] = []; - setLoaded(null); - setError(''); - setSlotId(''); - setMediaError(false); - void (async () => { - try { - const bundle = published - ? storedFidelityBundle(published) - : await loadFidelityBundle(reader!, runId); - if (cancelled) return; - const urls = new Map(published?.assets.map(([path, asset]) => [path, asset.url])); - if (!published) - for (const [path, blob] of bundle.files) { - const url = URL.createObjectURL( - new Blob([blob], { - type: path.endsWith('.mp4') ? 'video/mp4' : 'application/octet-stream', - }), - ); - created.push(url); - urls.set(path, url); - } - const raw = await bundle.files.get('report/index.html')?.text(); - if (cancelled) return; - setLoaded({ - bundle, - urls, - downloads: published - ? new Map(published.assets.map(([path, asset]) => [path, asset.downloadUrl])) - : urls, - html: raw ? renderReportHtml(raw, urls) : '', - }); - } catch (error) { - if (!cancelled) setError(error instanceof Error ? error.message : String(error)); - } - })(); - return () => { - cancelled = true; - created.forEach(URL.revokeObjectURL); - }; - }, [reader, published, runId]); - const fmt = (value: Json | undefined) => - typeof value === 'number' - ? value.toLocaleString('en-US', { maximumFractionDigits: 4 }) - : value === true - ? s.yes - : value === false - ? s.no - : value === null || value === undefined || value === '' - ? s.unavailable - : typeof value === 'string' - ? value - : JSON.stringify(value); - if (loadError) - return ( - - {s.error} -

{loadError}

-
- ); - if (!loaded) return {s.loading}; - const { bundle, urls, downloads } = loaded; - const comparison = bundle.comparison; - const slots = rows(at(bundle.portable, 'slots')); - const slot = slots.find((item) => at(item, 'slot_id') === slotId) ?? slots[0]; - const metrics = at(slot, 'metrics'); - const table = (values: [string, Json | undefined][]) => ( -
- {values.map(([label, value]) => ( -
-
{label}
-
{fmt(value)}
-
- ))} -
- ); - return ( -
- -
- {s.title} -

{s.intro}

-
-
- {[ - [s.pairs, at(comparison, 'summary', 'matched_valid_pairs')], - [ - s.outside, - slots.filter((item) => - rows(at(item, 'checks')).some( - (check) => - text(at(check, 'name')).startsWith('fidelity.') && - at(check, 'status') === 'fail', - ), - ).length, - ], - [s.inconclusive, at(comparison, 'summary', 'inconclusive_slots')], - ].map(([label, value]) => ( -
-

{String(label)}

-

{fmt(value)}

-
- ))} -
- {table([ - [ - s.calibration, - at(comparison, 'policy', 'calibration_status') === 'uncalibrated' - ? s.uncalibrated - : at(comparison, 'policy', 'calibration_status'), - ], - [s.qualification, at(comparison, 'release_qualified')], - ])} -

{s.note}

-

{s.runtimeNote}

-
- -
- { - setSlotId(value); - setMediaError(false); - track('video_fidelity_clip_selected', { slot: value }); - }} - options={slots.map((item) => ({ - value: text(at(item, 'slot_id')), - label: `${text(at(item, 'slot_id'))} · ${text(at(item, 'case_id'))}`, - }))} - /> -
-

- {s.prompt}: - {text(at(slot, 'prompt'))} -

-

- {s.seed}: {fmt(at(slot, 'seed'))} · {s.audio} -

-
- {ROLES.map((role) => { - const observation = at(slot, role); - const path = text(at(observation, 'artifact_path')); - const source = fidelitySource(bundle, role); - const sourceId = at(source, 'ci', 'databaseId'); - const ci = bundle.documents.get(`sources/${sourceId}/ci.json`); - return ( -
- - {s[role]} · {fmt(at(ci, 'site', 'gpu_model'))} - - {urls.get(`report/${path}`) ? ( -
- ); - })} -
- {mediaError &&

{s.mediaError}

} -

{s.timing}

-
- {s.details} -
-            {JSON.stringify(
-              {
-                generation: at(comparison, 'plan', 'generation'),
-                baseline: at(comparison, 'baseline', 'configuration'),
-                candidate: at(comparison, 'candidate', 'configuration'),
-              },
-              null,
-              2,
-            )}
-          
-
-
- - {s.metrics} - {table([ - [ - s.psnr, - at(metrics, 'video_identical') === true ? s.identical : at(metrics, 'video_psnr_db'), - ], - [s.mae, at(metrics, 'video_mae')], - [s.spectral, at(metrics, 'audio_spectral_cosine')], - [s.rms, at(metrics, 'audio_rms_ratio')], - [s.waveform, at(metrics, 'audio_waveform_mae')], - [ - s.frames, - `${fmt(at(metrics, 'video_compared_frames'))} / ${fmt(at(metrics, 'video_total_frames'))}`, - ], - [s.videoCoverage, at(metrics, 'video_sample_coverage_fraction')], - [s.audioCoverage, at(metrics, 'audio_sample_coverage_fraction')], - [s.audioSamples, at(metrics, 'audio_compared_samples_per_channel')], - ])} -
- {s.checks} -
- - - - {[s.check, s.status, s.observed, s.threshold, s.unit].map((label) => ( - - ))} - - - - {rows(at(slot, 'checks')).map((check, index) => ( - - {['name', 'status', 'observed', 'threshold', 'unit'].map((key) => ( - - ))} - - ))} - -
- {label} -
- {fmt(at(check, key))} -
-
-
-
- - {s.sources} -
- {ROLES.map((role) => { - const source = fidelitySource(bundle, role); - const id = at(source, 'ci', 'databaseId'); - const artifact = at(source, 'artifact', 'id'); - return ( - - ); - })} -
- - {s.producer} #{runId} - - {table([[s.digest, bundle.comparisonSha256]])} - {downloads.get('comparison.json') && ( - - {s.raw} - - )} - {loaded.html && ( -
setReportOpen(event.currentTarget.open)}> - {s.report} - {reportOpen && ( -