File tree Expand file tree Collapse file tree
apps/dreamverse/scripts/modal Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -111,9 +111,12 @@ DREAMVERSE_MAX_AUTOTUNE=1 \
111111
112112### Autoscaling and cost safety
113113
114- The current deployed script is capped with ` max_containers=1 ` . If uncapped, concurrent requests could
115- spawn multiple containers, while ` max_containers=1 ` queued requests onto one
116- container instead of multiplying B200 cost.
114+ The current deployed script keeps exactly one B200 container warm by setting
115+ ` min_containers=1 ` and ` max_containers=1 ` . ` min_containers=1 ` prevents Modal
116+ from scaling the deployment down to zero after idle periods, which avoids paying
117+ the expensive torch-compile/startup-warmup cost again on the next request.
118+ ` max_containers=1 ` caps concurrency so concurrent requests queue onto the warm
119+ container instead of spawning additional B200 containers and multiplying cost.
117120
118121### Common gotchas
119122
Original file line number Diff line number Diff line change 3939 memory = 65536 ,
4040 timeout = 7200 ,
4141 startup_timeout = 4800 ,
42+ min_containers = 1 ,
4243 max_containers = 1 ,
4344 secrets = [modal .Secret .from_name ("dreamverse-api-keys" )],
4445 volumes = {
You can’t perform that action at this time.
0 commit comments