Skip to content

Commit f74fdc9

Browse files
[misc]: surface GenerationResult.peak_memory_mb in the MiniMax H3 examples
Every generation already computes its own peak. The worker calls torch.cuda.max_memory_allocated() at multiproc_executor.py:699 and the value reaches the caller as GenerationResult.peak_memory_mb, but both H3 examples print only the generation time and drop it. On a memory-constrained device that number is the one that decides whether a configuration runs at all, and it is what a reader has to reproduce to check any memory claim about this model. Right now they cannot: the field is populated, handed over, and discarded. examples/inference/optimizations/spark_benchmark.py already prints it, so this only brings the H3 examples in line. Two lines each, guarded on the value being present, so nothing changes for a backend that does not report it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent a159b63 commit f74fdc9

2 files changed

Lines changed: 7 additions & 0 deletions

File tree

examples/inference/basic/basic_fasth3.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -343,6 +343,9 @@ def run(args: argparse.Namespace) -> list[float]:
343343
generation_time = getattr(result, "generation_time", None)
344344
if generation_time is not None:
345345
print(f"Generation time: {float(generation_time):.3f}s")
346+
peak_memory_mb = getattr(result, "peak_memory_mb", None)
347+
if peak_memory_mb is not None:
348+
print(f"Peak memory: {float(peak_memory_mb):.1f} MB")
346349
denoise_time = _denoise_seconds(result)
347350
if denoise_time is not None:
348351
measured_denoise_times.append(denoise_time)

examples/inference/basic/basic_minimax_h3_t2v.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,10 +114,14 @@ def main() -> None:
114114
# machine-readable: benchmark harnesses parse this line to separate
115115
# generation from model-load time (last occurrence = steady state)
116116
print(f"Generation time: {result.generation_time:.2f}s")
117+
if result.peak_memory_mb is not None:
118+
print(f"Peak memory: {result.peak_memory_mb:.1f} MB")
117119
for _ in range(args.repeats - 1):
118120
result = generator.generate(request)
119121
if result.generation_time is not None:
120122
print(f"Generation time: {result.generation_time:.2f}s")
123+
if result.peak_memory_mb is not None:
124+
print(f"Peak memory: {result.peak_memory_mb:.1f} MB")
121125
finally:
122126
generator.shutdown()
123127

0 commit comments

Comments
 (0)