Skip to content

Commit 770d2df

Browse files
committed
[docs]: drop leftover sequential-start wording from lazy-load offloading docs
The MLX phase-order sentence and "start a new generator" instruction belong to a different design. A later generate() reloads in-process.
1 parent bd5ee4c commit 770d2df

1 file changed

Lines changed: 6 additions & 7 deletions

File tree

docs/inference/offloading.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,12 @@ runs conditioning, and releases that encoder before it loads the DiT. After
2828
denoise it releases the DiT, then loads the video VAE for decode. Input
2929
preparation and unpatchify read geometry from checkpoint `config.json` (VAE
3030
spatial ratio / latent channels, DiT patch size) so those stages do not
31-
materialize weights just to read two integers. The MLX FastH3 runtime uses the
32-
same phase order. When host offload is off, DiT safetensors are read onto the
33-
accelerator instead of CPU-then-copy. A later `generate()` on the same worker
34-
reloads a released component from disk; start a new generator for a new prompt
35-
until prompt-cache reload exists. The flag is auto-enabled on unified-memory
36-
devices. Pass `--no-lazy-module-load` (or `lazy_module_load=False`) to keep every
37-
component resident.
31+
materialize weights just to read two integers. When host offload is off, DiT
32+
safetensors are read onto the accelerator instead of CPU-then-copy. A later
33+
`generate()` on the same worker reloads a released component from disk in
34+
process; it does not need a new generator. The flag is auto-enabled on
35+
unified-memory devices. Pass `--no-lazy-module-load` (or `lazy_module_load=False`)
36+
to keep every component resident.
3837

3938
## Behavior Explanation
4039

0 commit comments

Comments
 (0)