Skip to content

Commit 8757b01

Browse files
Mister-RaggsSolitaryThinker
authored andcommitted
[docs]: pin GPU SKU + final empirical numbers in PORT_STATUS
Component parity validated on Tongyi-MAI/Z-Image-Turbo on NVIDIA A40 (46068 MiB, driver 565.57.01) as of 2026-05-12. All thresholds pass with 1.5-1.6x headroom over observed: last_hidden_state bf16 mean (worst): 0.0168 < 0.025 last_hidden_state bf16 median (worst): 0.0127 < 0.020 hidden_states[-2] bf16 mean (worst): 0.0739 < 0.120 hidden_states[-2] bf16 median (worst): 0.0625 < 0.100 Handoff notes updated to reflect that component parity is done; next port-stack steps (DiT, pipeline, conversion, SSIM) are scoped to a future PR, not hao-ai-lab#1339.
1 parent 92ca106 commit 8757b01

1 file changed

Lines changed: 10 additions & 10 deletions

File tree

tests/local_tests/zimage/PORT_STATUS.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -37,12 +37,12 @@
3737
## Parity Commands
3838
| Scope | Command | Last Result | Notes |
3939
|---|---|---|---|
40-
| Scheduler | `pytest tests/local_tests/zimage/test_zimage_scheduler_parity.py -v -s` | PASS (2/2) on Z-Image-Turbo, validated 2026-05-12 | full `scheduler_config.json` now forwarded (was 3 keys) |
41-
| Tokenizer | `pytest tests/local_tests/zimage/test_zimage_tokenizer_parity.py -v -s` | PASS (2/2) on Z-Image-Turbo, validated 2026-05-12 | tokenizer resolves to `Qwen2TokenizerFast`; `apply_chat_template` parity included |
42-
| VAE decode | `pytest tests/local_tests/zimage/test_zimage_vae_parity.py -v -s` | PASS (1/1) on Z-Image-Turbo, validated 2026-05-12 | encode-path deferred |
43-
| Text encoder fp32 | `pytest tests/local_tests/zimage/test_zimage_encoder_parity.py::test_zimage_qwen3_encoder_parity_forward[fp32]` | PASS, validated 2026-05-12 | bit-exact (`last_hidden_state` max=0.0000, `hidden_states[-2]` max=0.0012) — FastVideo Qwen3 port is numerically correct |
44-
| Text encoder bf16 | `pytest tests/local_tests/zimage/test_zimage_encoder_parity.py::test_zimage_qwen3_encoder_parity_forward[bf16]` | PASS after threshold calibration, validated 2026-05-12 | empirical: `last_hidden_state` mean=0.0152 median=0.0117, `hidden_states[-2]` mean=0.0754 median=0.0625. Per-layer diag confirms monotonic accumulation across 35 layers, no single-layer spike — textbook bf16-tail signature |
45-
| Per-layer bf16 diag | `pytest tests/local_tests/zimage/test_zimage_encoder_parity.py::test_zimage_qwen3_encoder_per_layer_bf16_diagnostic -v -s` | PASS (informational only), validated 2026-05-12 | prints 37 hidden-state diffs (embedding + 35 layers + post-norm) for future debugging |
40+
| Scheduler | `pytest tests/local_tests/zimage/test_zimage_scheduler_parity.py -v -s` | PASS (2/2) on Z-Image-Turbo, A40, 2026-05-12 | full `scheduler_config.json` now forwarded (was 3 keys) |
41+
| Tokenizer | `pytest tests/local_tests/zimage/test_zimage_tokenizer_parity.py -v -s` | PASS (2/2) on Z-Image-Turbo, A40, 2026-05-12 | tokenizer resolves to `Qwen2TokenizerFast`; `apply_chat_template` parity included |
42+
| VAE decode | `pytest tests/local_tests/zimage/test_zimage_vae_parity.py -v -s` | PASS (1/1) on Z-Image-Turbo, A40, 2026-05-12 | encode-path deferred |
43+
| Text encoder fp32 | `pytest tests/local_tests/zimage/test_zimage_encoder_parity.py::test_zimage_qwen3_encoder_parity_forward[fp32]` | PASS on Z-Image-Turbo, A40, 2026-05-12 | bit-exact (`last_hidden_state` max=0.0000, `hidden_states[-2]` max=0.0012) across both batches — FastVideo Qwen3 port is numerically correct |
44+
| Text encoder bf16 | `pytest tests/local_tests/zimage/test_zimage_encoder_parity.py::test_zimage_qwen3_encoder_parity_forward[bf16]` | PASS on Z-Image-Turbo, A40, 2026-05-12 | empirical (worst across 2 batches): `last_hidden_state` mean=0.0168 median=0.0127 (thresholds 0.025 / 0.020, 1.5x headroom); `hidden_states[-2]` mean=0.0739 median=0.0625 (thresholds 0.120 / 0.100, 1.6x headroom). Per-layer diag confirms monotonic accumulation across 35 layers, no single-layer spike — textbook bf16-tail signature |
45+
| Per-layer bf16 diag | `pytest tests/local_tests/zimage/test_zimage_encoder_parity.py::test_zimage_qwen3_encoder_per_layer_bf16_diagnostic -v -s` | PASS (informational only) on Z-Image-Turbo, A40, 2026-05-12 | prints 37 hidden-state diffs (embedding + 35 layers + post-norm) for future debugging |
4646

4747
## Open Questions
4848
| ID | Question | Owner | Needed By Phase | Status | Resolution |
@@ -67,11 +67,11 @@
6767
## Decisions
6868
| Date | Decision | Rationale | Impact |
6969
|---|---|---|---|
70-
| 2026-05-12 | bf16 encoder parity uses distribution checks (mean + median) instead of element-wise `assert_close`; thresholds calibrated to empirical Z-Image-Turbo numbers + 1.6x headroom. | Z-Image-Turbo's Qwen3 text encoder is 35 layers (not the 24 originally assumed). Cross-kernel bf16 (FastVideo's fused QKVParallel + MergedColumnParallel + SiluAndMul vs HF's unfused equivalents) accumulates into a long max tail (~4.0 at layer 34) but median stays low (0.06). Per-layer diagnostic test confirmed growth is smooth and monotonic with no single-layer spike — textbook bf16-tail signature, fp32 is bit-exact. Element-wise `assert_close` is meaningless on this profile; mean + median + the per-layer diag together detect real bugs (which push mean ≫ atol AND median > 0.01). | Final assertion shape: `last_hidden_state` mean < 0.025, median < 0.020; `hidden_states[-2]` mean < 0.120, median < 0.100. |
70+
| 2026-05-12 | bf16 encoder parity uses distribution checks (mean + median) instead of element-wise `assert_close`; thresholds calibrated to empirical Z-Image-Turbo numbers on A40 + 1.5–1.6x headroom. | Z-Image-Turbo's Qwen3 text encoder is 35 layers (not the 24 originally assumed). Cross-kernel bf16 (FastVideo's fused QKVParallel + MergedColumnParallel + SiluAndMul vs HF's unfused equivalents) accumulates into a long max tail (~4.0 at layer 34) but median stays low (0.06). Per-layer diagnostic test confirmed growth is smooth and monotonic with no single-layer spike — textbook bf16-tail signature, fp32 is bit-exact. Element-wise `assert_close` is meaningless on this profile; mean + median + the per-layer diag together detect real bugs (which push mean ≫ atol AND median > 0.01). | Final assertion shape: `last_hidden_state` mean < 0.025, median < 0.020; `hidden_states[-2]` mean < 0.120, median < 0.100. Validated on NVIDIA A40 (driver 565.57.01, 46068 MiB) 2026-05-12. |
7171
| 2026-05-12 | `AutoModel.from_pretrained` uses `dtype=` (not `torch_dtype=`). | transformers 4.57.3 emits `torch_dtype is deprecated! Use dtype instead!` warning. Mrinaald's original `dtype=` kwarg was correct; the temporary switch to `torch_dtype=` (in response to a Copilot review comment) was reverted. ||
7272
| 2026-05-12 | Scheduler parity forwards the full `scheduler_config.json` dict (minus Diffusers loader keys), not 3 hand-picked keys. | Future on-disk fields (`time_shift_type`, `invert_sigmas`, etc.) would have been silently dropped. | Makes parity reflect the actual on-disk config. |
7373

7474
## Handoff Notes
75-
- Next agent should resolve Q001 (pin Z-Image SHA), then move to transformer port (I004).
76-
- Loader-side strictness is now contract-asserted in the encoder parity test; do not relax `ALLOWED_UNEXPECTED_KEYS` without updating the test.
77-
- Pipeline preset wiring must also pin `use_reference_discrete_timesteps=True` in `scheduler_config.json` (I002).
75+
- Component parity (scheduler / tokenizer / VAE / Qwen3 fp32 + bf16) is **fully validated** on `Tongyi-MAI/Z-Image-Turbo` weights on A40 as of 2026-05-12.
76+
- Loader-side strictness is contract-asserted in the encoder parity test; do not relax `ALLOWED_UNEXPECTED_KEYS` without updating the test.
77+
- Next port-stack steps (separate PR, not in #1339 scope): `ZImageTransformer2DModel` port (I004), pipeline preset including `use_reference_discrete_timesteps=True` pinned in `scheduler_config.json` (I002), conversion-or-direct-load story, SSIM media regression (blocked on PR #1321's `media_extension` helper landing for T2I `.png` output).

0 commit comments

Comments
 (0)