Skip to content

Fix stop cutting the emulator's shutdown short - #551

Open
paolosalvatori wants to merge 3 commits into
mainfrom
smf-948-wait-for-the-emulator-shutdown
Open

paolosalvatori wants to merge 3 commits into
mainfrom
smf-948-wait-for-the-emulator-shutdown

Conversation

@paolosalvatori

@paolosalvatori paolosalvatori commented Oct 15, 2026 •

Copy link
Copy Markdown

Motivation

lstk creates the emulator container without a stop timeout, so lstk stop gets Docker's default: SIGTERM, then a kill 10 seconds later. The Azure emulator needs longer than that to remove the containers it started for Azure resources (k3d clusters, databases, storage backends), and after lstk stop they kept running. LocalStack's supervisor gives the emulator SHUTDOWN_TIMEOUT seconds to shut down (5 by default, and the Azure image is raising it to 60), but whatever that value, Docker kills the container after 10 seconds.

NOTE: if you don't want to change the way lstk stop works for all the emulators, can we at least increase the timeout for the Azure emulator to a value higher than 10 seconds (e.g. 30 seconds)?

Fixes SMF-948

Solution

  • At start, lstk sets the container's Docker stop timeout to the emulator's SHUTDOWN_TIMEOUT plus 5 seconds, so the supervisor completes the shutdown, or its own kill, before Docker kills the container. lstk reads the value as the container will: the container env (a profile's env) over the image env, and a LOCALSTACK_SHUTDOWN_TIMEOUT (from the shell or a profile) in its place, as the image's entrypoint re-exports it. Without one, the supervisor's default of 5 seconds gives 10, Docker's default, so an image without the variable stops exactly as before.
  • A SHUTDOWN_TIMEOUT of 0 or less tells the supervisor never to kill the emulator: lstk then gives the container a bounded 300 seconds rather than Docker's wait forever, and so it does for a container whose own stop timeout says to wait forever.
  • lstk stop passes the stop timeout to Docker and waits that long plus 15 seconds, and at least 30 (the previous fixed wait). A container without a stop timeout of its own (created by an earlier lstk version or outside lstk), or with a shorter one (Podman reports its 10-second default for every container), gets the one derived from its environment the same way.
Docs

User-facing:

  • lstk stop can take longer: up to the emulator's SHUTDOWN_TIMEOUT plus 5 seconds before Docker kills the container, 65 seconds with an image that sets 60.
  • docker stop on a container lstk started, and the Docker daemon's own shutdown, wait as long, as the timeout is part of the container's configuration.
  • A SHUTDOWN_TIMEOUT or LOCALSTACK_SHUTDOWN_TIMEOUT set in a profile's env, or a LOCALSTACK_SHUTDOWN_TIMEOUT exported in the shell, now also sets how long Docker waits for the container to stop; 0 or less means 300 seconds.

Worth a line in the lstk stop docs, and wherever SHUTDOWN_TIMEOUT is documented for lstk users.

Review

Human review advised: it changes how long lstk stop can take for every emulator image, and it was not discussed with the lstk team beforehand.

Unchanged, and now more likely to be seen: Ctrl+C during a stop prints "failed to stop LocalStack: context canceled" while Docker goes on stopping the container.

Towards SMF-948

馃 Generated with Claude Code

@paolosalvatori paolosalvatori added semver: patch docs: needed Pull request requires documentation updates labels Oct 15, 2026
@paolosalvatori
paolosalvatori marked this pull request as ready for review October 15, 2026 16:56
Copilot AI balanced review requested due to automatic review settings October 15, 2026 16:56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃數 Needs a closer look

Shutdown timing changes across emulator images and Docker-compatible runtimes warrant human validation.

0 open findings

What changed in this PR

Aligns Docker鈥檚 stop timeout with the emulator鈥檚 shutdown budget so cleanup can finish before Docker kills it.

Changes:

  • Derives stop timeouts from environment settings and stores them at container creation.
  • Extends stop waits and handles older or externally created containers.
  • Adds unit and integration coverage for timeout propagation and graceful shutdown.
File Description
test/鈥媔ntegration/鈥媠top_timeout_test.go Tests timeout configuration and graceful shutdown.
test/鈥媔ntegration/鈥媘ain_test.go Extracts reusable test-image creation helper.
internal/鈥媟untime/鈥媟untime.go Adds timeout fields and stop parameter.
internal/鈥媟untime/鈥媘ock_runtime.go Updates generated stop mocks.
internal/鈥媟untime/鈥媏xternal.go Adapts unsupported stop implementation.
internal/鈥媟untime/鈥媏xternal_test.go Updates stop interface test.
internal/鈥媟untime/鈥媎ocker.go Stores, reads, and passes Docker stop timeouts.
internal/鈥媟untime/鈥媎ocker_stop_timeout_test.go Tests Docker API timeout handling.
internal/鈥媍ontainer/鈥媡elemetry_test.go Updates lifecycle test expectations.
internal/鈥媍ontainer/鈥媠top.go Applies resolved timeout and wait budget.
internal/鈥媍ontainer/鈥媠top_timeout.go Implements timeout derivation and bounds.
internal/鈥媍ontainer/鈥媠top_timeout_test.go Tests environment precedence and fallback values.
internal/鈥媍ontainer/鈥媠top_test.go Tests stop timeout selection and deadlines.
internal/鈥媍ontainer/鈥媠tart.go Resolves timeout after image pull.
internal/鈥媍ontainer/鈥媠tart_test.go Updates startup timeout expectations.
Files not reviewed (1)
  • internal/runtime/mock_runtime.go: Generated file

馃 Review effort: Balanced


馃挕 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@gtsiolis gtsiolis left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UX LGTM. Thanks, @paolosalvatori!

Tested both binaries with real AKS and storage resources.

Binary Result
Using lstk 1.4.1 Reported success, but emulator exited 137.
Rebuilt ./bin/lstk Emulator exited cleanly with 0; resource containers and network removed before success

paolosalvatori and others added 3 commits October 16, 2026 19:42
Co-Authored-By: Claude <noreply@anthropic.com>
The stop timeout now follows the SHUTDOWN_TIMEOUT the supervisor reads: a LOCALSTACK_SHUTDOWN_TIMEOUT, from the host or a profile, replaces it, as the image's entrypoint re-exports it, and the container's value wins over the image's even when it is 0.
A SHUTDOWN_TIMEOUT of 0 or less tells the supervisor never to kill the emulator, so lstk gives that container a bounded 300 s instead of ignoring the value.
lstk stop passes the stop timeout to Docker: a container without one of its own (created by an older lstk, or outside lstk) gets one derived from its environment instead of Docker's 10 s, and one that would wait forever gets the bounded 300 s.
Integration tests stop a stand-in that takes 12 s to exit after its stop signal, once started by lstk and once outside it, and require exit code 0 instead of Docker's kill.

Co-Authored-By: Claude <noreply@anthropic.com>
Podman reports a stop timeout for every container, so the environment's must act as a floor rather than a fallback.

Co-Authored-By: Claude <noreply@anthropic.com>
@paolosalvatori
paolosalvatori force-pushed the smf-948-wait-for-the-emulator-shutdown branch from e372d08 to 85b6a57 Compare October 16, 2026 17:42
@paolosalvatori

Copy link
Copy Markdown
Author

UX LGTM. Thanks, @paolosalvatori!

Tested both binaries with real AKS and storage resources.

Binary Result
Using lstk 1.4.1 Reported success, but emulator exited 137.
Rebuilt ./bin/lstk Emulator exited cleanly with 0; resource containers and network removed before success

Thanks for the test and review @gtsiolis 馃檹

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs: needed Pull request requires documentation updates semver: patch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants