Skip to content

Commit bd3bf3b

Browse files
committed
refactor(snapstart): drop the redundant pre-snapshot readiness gate
With AWS_LWA_ASYNC_INIT ignored under SnapStart, check_init_health takes its blocking path and `main` awaits it before `run()` — so the application is already ready by the time before_snapshot runs. The gate only duplicated that invariant in a second place. The one case it still covered was a library consumer calling Adapter::new() then run() while skipping check_init_health(). That is not a coherent guarantee for them anyway: it would protect the snapshot while leaving their invocations unguarded, since they also skip ready_at_init and so never get the first-request re-check. If someone re-enables async_init under SnapStart, the test on effective_async_init fails, which is an earlier and louder signal than a runtime wait. Removes the gate, the two tests written for it, and a duplicate happy-path test it left behind. `ensure_ready`'s `phase` parameter goes too — after_restore is now its only caller, so it was a parameter with one possible value. Docs reconciled with the removal: the guide's lifecycle step 1 no longer claims a pre-snapshot wait (initialization already did it), AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS bounds two waits again rather than three, and the CHANGELOG's "hooks firing before the application is ready" fix folds into the async_init entry, which is where the actual fix lives.
1 parent 85e223d commit bd3bf3b

3 files changed

Lines changed: 23 additions & 118 deletions

File tree

CHANGELOG.md

Lines changed: 7 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,11 @@
2424
app. Default: 4 seconds. A value that is set but unusable falls back to the
2525
default and logs a warning.
2626
- Add `AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` to bound the readiness check
27-
(fractional seconds allowed, e.g. `0.5`), applied to the initial cold-start
28-
readiness wait, the pre-snapshot wait, and the post-SnapStart-restore readiness
29-
check. When set and the app does not become ready within it, the adapter
30-
**refuses to serve**: cold-start init fails (the runtime never starts), and taking
31-
a snapshot or completing a restore fails, rather than admitting traffic to an
32-
app that never reported ready. When unset (the default) the wait is
27+
(fractional seconds allowed, e.g. `0.5`), applied to both the initial cold-start
28+
readiness wait and the post-SnapStart-restore readiness check. When set and the app
29+
does not become ready within it, the adapter **refuses to serve**: cold-start init
30+
fails (the runtime never starts) and a restore fails, rather than admitting traffic
31+
to an app that never reported ready. When unset (the default) the wait is
3332
**unbounded**, matching the previous behavior, so existing slow-cold-start apps
3433
are unaffected unless they opt in. On-demand cold starts using `async_init` keep
3534
that path's own fixed ~9.8s bound (non-fatal) and are not affected by this variable.
@@ -45,14 +44,6 @@
4544
is normalized so it behaves like `/api`. **Upgrade note:** this changes the path
4645
forwarded to your app for those inputs — deployments that relied on the old
4746
repeated/partial stripping should verify their routes.
48-
- Fix the hooks firing before the application is ready. With `AWS_LWA_ASYNC_INIT=true`
49-
the adapter finishes initialization after 9.8 seconds even if the app has not bound
50-
its port yet, so the snapshot could capture a still-booting app and each hook `POST`
51-
then failed immediately with a connection error. The adapter now waits for the
52-
readiness check before the snapshot is taken — bounded by
53-
`AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` when it is set — which covers both hooks,
54-
since the after-restore hook deliberately runs before its own readiness check so the
55-
application can reconnect before its health is judged.
5647
- `AWS_LWA_ASYNC_INIT` is now ignored under SnapStart and Provisioned Concurrency, with
5748
a warning. It exists to work around the short initialization limit for on-demand cold
5849
starts by reporting init complete before the application is ready; neither of those
@@ -63,7 +54,8 @@
6354
`AWS_LWA_ASYNC_INIT=true` together with SnapStart or provisioned concurrency now waits
6455
for its readiness check during initialization instead of finishing early; bound that
6556
wait with `AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` if you want it to fail rather than
66-
block.
57+
block. Under SnapStart this is also what guarantees the snapshot is taken of a fully
58+
initialized application.
6759

6860
### Dependencies
6961

docs/guide/src/features/snapstart.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,13 @@ Both hooks are opt-in and independent — each fires only when its variable is s
1515

1616
The adapter always registers for the SnapStart lifecycle; the Lambda runtime invokes the hooks only when your function runs under SnapStart. When it does, the adapter participates as follows:
1717

18-
1. **Before checkpoint** — the adapter first waits for your application to pass the readiness check, so a snapshot is never taken of a still-booting app. Then, if `AWS_LWA_SNAPSTART_BEFORE_CHECKPOINT_PATH` is set, it sends an empty `POST` to that path, and signals Lambda that it is ready for the snapshot.
18+
1. **Before checkpoint**if `AWS_LWA_SNAPSTART_BEFORE_CHECKPOINT_PATH` is set, the adapter sends an empty `POST` to that path on your application, then signals Lambda that it is ready for the snapshot. Initialization has already waited for your application to report ready by this point, so the snapshot is never taken of a still-booting app.
1919

2020
2. **After restore** — Lambda restores the environment. The adapter first refreshes its own HTTP connection to your application (so it never reuses a connection captured in the snapshot); then, if `AWS_LWA_SNAPSTART_AFTER_RESTORE_PATH` is set, sends an empty `POST` to that path; and finally re-runs the readiness check before admitting traffic.
2121

2222
> **Note:** `AWS_LWA_ASYNC_INIT` is ignored under SnapStart (and under Provisioned Concurrency). It works around the short initialization limit for on-demand cold starts by reporting init complete before the application is ready; neither of those environments has that limit, and finishing early would snapshot — or serve — a half-initialized application. The adapter logs a warning when it ignores the setting.
2323
24-
Each hook is an empty `POST`, and your application must respond with a `2xx` status. A non-`2xx` response or a connection failure fails the SnapStart phase — initialization for the before-checkpoint hook, restore for the after-restore hook — rather than serving traffic against an improperly prepared application. The adapter does not impose its own deadline on a hook: Lambda already bounds both phases, and the after-restore hook in particular must complete within your function timeout, so keep that in mind when a hook does slow work such as draining a large connection pool. The final readiness check runs on every restore (whether or not an after-restore path is configured). By default this readiness wait is unbounded; set `AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` (fractional seconds allowed, e.g. `0.5`) to bound it, in which case a restore whose application does not report ready within that timeout fails. The same variable also bounds the initial cold-start readiness check and the pre-snapshot wait in step 1: when set and the application does not report ready within the timeout, initialization fails (the Lambda runtime never starts) rather than serving traffic — or snapshotting — an app that never came up.
24+
Each hook is an empty `POST`, and your application must respond with a `2xx` status. A non-`2xx` response or a connection failure fails the SnapStart phase — initialization for the before-checkpoint hook, restore for the after-restore hook — rather than serving traffic against an improperly prepared application. The adapter does not impose its own deadline on a hook: Lambda already bounds both phases, and the after-restore hook in particular must complete within your function timeout, so keep that in mind when a hook does slow work such as draining a large connection pool. The final readiness check runs on every restore (whether or not an after-restore path is configured). By default this readiness wait is unbounded; set `AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` (fractional seconds allowed, e.g. `0.5`) to bound it, in which case a restore whose application does not report ready within that timeout fails. The same variable also bounds the initial cold-start readiness check, which under SnapStart is what guarantees the application is ready before the snapshot: when set and the application does not report ready within the timeout, initialization fails (the Lambda runtime never starts) rather than snapshotting or serving an app that never came up.
2525

2626
## Why you need the hooks
2727

src/snapstart.rs

Lines changed: 14 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -119,16 +119,11 @@ impl SnapStartHooks {
119119
impl SnapStartResource for SnapStartHooks {
120120
fn before_snapshot(&self) -> BoxFuture<'_, Result<(), Error>> {
121121
Box::pin(async move {
122-
// Wait for the app unconditionally, before any hook, so a snapshot is
123-
// never taken of a still-booting app. `check_init_health` already blocks
124-
// until ready under SnapStart (AWS_LWA_ASYNC_INIT is ignored there, see
125-
// `effective_async_init`), so this is defense in depth — it also covers a
126-
// consumer that drives the `Service` impl without calling
127-
// `check_init_health`. It has to be unconditional: `after_restore` POSTs
128-
// before its own readiness check (step 2 before step 3, deliberately), so
129-
// a half-booted snapshot would fail every restore even with no
130-
// before-checkpoint hook configured.
131-
self.ensure_ready(&self.client, "before-checkpoint").await?;
122+
// No readiness wait here: the app is already ready. `AWS_LWA_ASYNC_INIT` is
123+
// ignored under SnapStart (see `effective_async_init`), so
124+
// `check_init_health` takes its blocking path and `main` awaits it before
125+
// `run()`. Re-checking would only duplicate that invariant in a second
126+
// place.
132127
if let Some(path) = self.before_checkpoint_path.as_deref() {
133128
Self::post_hook(&self.client, &self.domain, path).await?;
134129
}
@@ -160,24 +155,22 @@ impl SnapStartResource for SnapStartHooks {
160155
// 3. Confirm the app is serving again before traffic is admitted.
161156
// A configured timeout bounds the wait and fails the restore on
162157
// expiry; when unset the wait is unbounded (historical behavior).
163-
self.ensure_ready(&fresh, "after-restore").await?;
158+
self.ensure_ready(&fresh).await?;
164159

165160
Ok(())
166161
})
167162
}
168163
}
169164

170165
impl SnapStartHooks {
171-
/// Waits for the app to report ready, bounded by
172-
/// `AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` when set. `phase` names the SnapStart
173-
/// phase in the timeout error so an init failure is distinguishable from a restore
174-
/// failure.
166+
/// Waits for the app to report ready after a restore, bounded by
167+
/// `AWS_LWA_READINESS_CHECK_TIMEOUT_SECONDS` when set.
175168
///
176-
/// Unset means unbounded, which cannot fail — only block until Lambda's own phase
169+
/// Unset means unbounded, which cannot fail — only block until Lambda's own restore
177170
/// timeout, with the escalating `app is not ready after {}ms` log as the signal.
178-
async fn ensure_ready(&self, client: &Client<HttpConnector, Body>, phase: &str) -> Result<(), Error> {
171+
async fn ensure_ready(&self, client: &Client<HttpConnector, Body>) -> Result<(), Error> {
179172
match self.readiness_timeout {
180-
Some(t) => self.ensure_ready_with_timeout(client, t, phase).await,
173+
Some(t) => self.ensure_ready_with_timeout(client, t).await,
181174
None => {
182175
self.wait_ready(client).await;
183176
Ok(())
@@ -191,11 +184,10 @@ impl SnapStartHooks {
191184
&self,
192185
client: &Client<HttpConnector, Body>,
193186
readiness_timeout: Duration,
194-
phase: &str,
195187
) -> Result<(), Error> {
196188
timeout(readiness_timeout, self.wait_ready(client)).await.map_err(|_| {
197189
Error::from(format!(
198-
"SnapStart {phase} readiness check timed out after {readiness_timeout:?}"
190+
"SnapStart after-restore readiness check timed out after {readiness_timeout:?}"
199191
))
200192
})
201193
}
@@ -265,85 +257,8 @@ mod tests {
265257
m.assert();
266258
}
267259

268-
/// `before_snapshot` must gate the hook POST on readiness: an app that has not
269-
/// bound its port yet would fail the init phase with `ECONNREFUSED`.
270-
///
271-
/// The hook route here would answer 200, but readiness never passes — so the hook
272-
/// must not be called at all.
273-
#[tokio::test]
274-
async fn before_snapshot_waits_for_readiness_before_posting() {
275-
let server = MockServer::start();
276-
let hook = server.mock(|when, then| {
277-
when.method(httpmock::Method::POST).path("/before");
278-
then.status(200);
279-
});
280-
// Readiness target answers 503 forever, so the app is never ready.
281-
server.mock(|when, then| {
282-
when.path("/never-ready");
283-
then.status(503);
284-
});
285-
let mut h = hooks_with_health(&server, Some("/before"), None, "/never-ready");
286-
h.readiness_timeout = Some(Duration::from_millis(150));
287-
288-
let err = h
289-
.before_snapshot()
290-
.await
291-
.expect_err("an app that is not ready must fail the before-checkpoint phase");
292-
assert!(
293-
err.to_string().contains("before-checkpoint") && err.to_string().contains("readiness"),
294-
"error must name the before-checkpoint readiness check, got: {err}"
295-
);
296-
hook.assert_calls(0);
297-
}
298-
299-
/// The readiness gate must be UNCONDITIONAL, not tied to the before-checkpoint
300-
/// hook being configured.
301-
///
302-
/// The hooks are independent, so a function may set only
303-
/// `AWS_LWA_SNAPSTART_AFTER_RESTORE_PATH`. With the gate inside
304-
/// `if let Some(before_checkpoint_path)`, that configuration snapshots whatever
305-
/// state the app is in — and `after_restore` POSTs its hook before its own
306-
/// readiness check (step 2 before step 3, deliberately, so the reconnect happens
307-
/// before health is judged), so the POST would hit a process that is not listening
308-
/// and every restore would fail.
309-
#[tokio::test]
310-
async fn before_snapshot_waits_for_readiness_with_no_hook_configured() {
311-
let server = MockServer::start();
312-
server.mock(|when, then| {
313-
when.path("/never-ready");
314-
then.status(503);
315-
});
316-
// Only the AFTER-restore hook is configured.
317-
let mut h = hooks_with_health(&server, None, Some("/after"), "/never-ready");
318-
h.readiness_timeout = Some(Duration::from_millis(150));
319-
320-
let err = h
321-
.before_snapshot()
322-
.await
323-
.expect_err("the snapshot must not be taken against an app that is not ready");
324-
assert!(
325-
err.to_string().contains("before-checkpoint") && err.to_string().contains("readiness"),
326-
"error must name the before-checkpoint readiness check, got: {err}"
327-
);
328-
}
329-
330-
/// The gate must not change the happy path: a ready app still gets the POST.
331-
#[tokio::test]
332-
async fn before_snapshot_posts_once_app_is_ready() {
333-
let server = MockServer::start();
334-
let m = server.mock(|when, then| {
335-
when.method(httpmock::Method::POST).path("/before");
336-
then.status(200);
337-
});
338-
let h = hooks(&server, Some("/before"), None);
339-
assert!(h.before_snapshot().await.is_ok());
340-
m.assert();
341-
}
342-
343260
#[tokio::test]
344-
/// With no hook configured there is nothing to POST, but the readiness wait still
345-
/// runs (`hooks` mocks a healthy `/health`), so this is not a full no-op.
346-
async fn before_snapshot_posts_nothing_when_unset() {
261+
async fn before_snapshot_noop_when_unset() {
347262
let server = MockServer::start();
348263
let h = hooks(&server, None, None);
349264
assert!(h.before_snapshot().await.is_ok());
@@ -451,9 +366,7 @@ mod tests {
451366
let h = hooks_with_health(&server, None, None, "/never");
452367
let client = build_client(Duration::from_secs(4), Pooling::Enabled);
453368

454-
let result = h
455-
.ensure_ready_with_timeout(&client, Duration::from_millis(100), "after-restore")
456-
.await;
369+
let result = h.ensure_ready_with_timeout(&client, Duration::from_millis(100)).await;
457370

458371
let err = result.expect_err("unready app should fail the readiness check");
459372
assert!(err.to_string().contains("timed out"), "unexpected error: {err}");

0 commit comments

Comments
 (0)