Repository navigation
Fix stop cutting the emulator's shutdown short - #551
paolosalvatori wants to merge 3 commits into
Conversation
There was a problem hiding this comment.
馃數 Needs a closer look
Shutdown timing changes across emulator images and Docker-compatible runtimes warrant human validation.
0 open findings
What changed in this PR
Aligns Docker鈥檚 stop timeout with the emulator鈥檚 shutdown budget so cleanup can finish before Docker kills it.
Changes:
- Derives stop timeouts from environment settings and stores them at container creation.
- Extends stop waits and handles older or externally created containers.
- Adds unit and integration coverage for timeout propagation and graceful shutdown.
| File | Description |
|---|---|
test/鈥媔ntegration/鈥媠top_timeout_test.go |
Tests timeout configuration and graceful shutdown. |
test/鈥媔ntegration/鈥媘ain_test.go |
Extracts reusable test-image creation helper. |
internal/鈥媟untime/鈥媟untime.go |
Adds timeout fields and stop parameter. |
internal/鈥媟untime/鈥媘ock_runtime.go |
Updates generated stop mocks. |
internal/鈥媟untime/鈥媏xternal.go |
Adapts unsupported stop implementation. |
internal/鈥媟untime/鈥媏xternal_test.go |
Updates stop interface test. |
internal/鈥媟untime/鈥媎ocker.go |
Stores, reads, and passes Docker stop timeouts. |
internal/鈥媟untime/鈥媎ocker_stop_timeout_test.go |
Tests Docker API timeout handling. |
internal/鈥媍ontainer/鈥媡elemetry_test.go |
Updates lifecycle test expectations. |
internal/鈥媍ontainer/鈥媠top.go |
Applies resolved timeout and wait budget. |
internal/鈥媍ontainer/鈥媠top_timeout.go |
Implements timeout derivation and bounds. |
internal/鈥媍ontainer/鈥媠top_timeout_test.go |
Tests environment precedence and fallback values. |
internal/鈥媍ontainer/鈥媠top_test.go |
Tests stop timeout selection and deadlines. |
internal/鈥媍ontainer/鈥媠tart.go |
Resolves timeout after image pull. |
internal/鈥媍ontainer/鈥媠tart_test.go |
Updates startup timeout expectations. |
Files not reviewed (1)
- internal/runtime/mock_runtime.go: Generated file
馃 Review effort: Balanced
馃挕 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
UX LGTM. Thanks, @paolosalvatori!
Tested both binaries with real AKS and storage resources.
| Binary | Result |
|---|---|
| Using lstk 1.4.1 | Reported success, but emulator exited 137. |
| Rebuilt ./bin/lstk | Emulator exited cleanly with 0; resource containers and network removed before success |
Co-Authored-By: Claude <noreply@anthropic.com>
The stop timeout now follows the SHUTDOWN_TIMEOUT the supervisor reads: a LOCALSTACK_SHUTDOWN_TIMEOUT, from the host or a profile, replaces it, as the image's entrypoint re-exports it, and the container's value wins over the image's even when it is 0. A SHUTDOWN_TIMEOUT of 0 or less tells the supervisor never to kill the emulator, so lstk gives that container a bounded 300 s instead of ignoring the value. lstk stop passes the stop timeout to Docker: a container without one of its own (created by an older lstk, or outside lstk) gets one derived from its environment instead of Docker's 10 s, and one that would wait forever gets the bounded 300 s. Integration tests stop a stand-in that takes 12 s to exit after its stop signal, once started by lstk and once outside it, and require exit code 0 instead of Docker's kill. Co-Authored-By: Claude <noreply@anthropic.com>
Podman reports a stop timeout for every container, so the environment's must act as a floor rather than a fallback. Co-Authored-By: Claude <noreply@anthropic.com>
e372d08 to
85b6a57
Compare
Thanks for the test and review @gtsiolis 馃檹 |
Motivation
lstkcreates the emulator container without a stop timeout, solstk stopgets Docker's default: SIGTERM, then a kill 10 seconds later. The Azure emulator needs longer than that to remove the containers it started for Azure resources (k3dclusters, databases, storage backends), and afterlstk stopthey kept running. LocalStack's supervisor gives the emulatorSHUTDOWN_TIMEOUTseconds to shut down (5 by default, and the Azure image is raising it to 60), but whatever that value, Docker kills the container after 10 seconds.Fixes SMF-948
Solution
lstksets the container's Docker stop timeout to the emulator'sSHUTDOWN_TIMEOUTplus 5 seconds, so the supervisor completes the shutdown, or its own kill, before Docker kills the container.lstkreads the value as the container will: the container env (a profile'senv) over the image env, and aLOCALSTACK_SHUTDOWN_TIMEOUT(from the shell or a profile) in its place, as the image's entrypoint re-exports it. Without one, the supervisor's default of 5 seconds gives 10, Docker's default, so an image without the variable stops exactly as before.SHUTDOWN_TIMEOUTof 0 or less tells the supervisor never to kill the emulator:lstkthen gives the container a bounded 300 seconds rather than Docker's wait forever, and so it does for a container whose own stop timeout says to wait forever.lstk stoppasses the stop timeout to Docker and waits that long plus 15 seconds, and at least 30 (the previous fixed wait). A container without a stop timeout of its own (created by an earlier lstk version or outside lstk), or with a shorter one (Podman reports its 10-second default for every container), gets the one derived from its environment the same way.Docs
User-facing:
lstk stopcan take longer: up to the emulator'sSHUTDOWN_TIMEOUTplus 5 seconds before Docker kills the container, 65 seconds with an image that sets 60.docker stopon a container lstk started, and the Docker daemon's own shutdown, wait as long, as the timeout is part of the container's configuration.SHUTDOWN_TIMEOUTorLOCALSTACK_SHUTDOWN_TIMEOUTset in a profile'senv, or aLOCALSTACK_SHUTDOWN_TIMEOUTexported in the shell, now also sets how long Docker waits for the container to stop; 0 or less means 300 seconds.Worth a line in the
lstk stopdocs, and whereverSHUTDOWN_TIMEOUTis documented for lstk users.Review
Human review advised: it changes how long
lstk stopcan take for every emulator image, and it was not discussed with the lstk team beforehand.Unchanged, and now more likely to be seen: Ctrl+C during a stop prints "failed to stop LocalStack: context canceled" while Docker goes on stopping the container.
Towards SMF-948
馃 Generated with Claude Code