Skip to content

Commit dafe049

Browse files
committed
[bugfix] Keep Dreamverse Modal container warm
1 parent d43c166 commit dafe049

2 files changed

Lines changed: 7 additions & 3 deletions

File tree

apps/dreamverse/scripts/modal/README.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -111,9 +111,12 @@ DREAMVERSE_MAX_AUTOTUNE=1 \
111111

112112
### Autoscaling and cost safety
113113

114-
The current deployed script is capped with `max_containers=1`. If uncapped, concurrent requests could
115-
spawn multiple containers, while `max_containers=1` queued requests onto one
116-
container instead of multiplying B200 cost.
114+
The current deployed script keeps exactly one B200 container warm by setting
115+
`min_containers=1` and `max_containers=1`. `min_containers=1` prevents Modal
116+
from scaling the deployment down to zero after idle periods, which avoids paying
117+
the expensive torch-compile/startup-warmup cost again on the next request.
118+
`max_containers=1` caps concurrency so concurrent requests queue onto the warm
119+
container instead of spawning additional B200 containers and multiplying cost.
117120

118121
### Common gotchas
119122

apps/dreamverse/scripts/modal/modal_app.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
memory=65536,
4040
timeout=7200,
4141
startup_timeout=4800,
42+
min_containers=1,
4243
max_containers=1,
4344
secrets=[modal.Secret.from_name("dreamverse-api-keys")],
4445
volumes={

0 commit comments

Comments
 (0)