Integration test deflake attempt

[ravenblackx]

Commit Message: Integration test deflake attempt
Additional Description: The specific integration test this is targeting was test/extensions/filters/http/on_demand:on_demand_integration_test, which is flaky about 1/1000. An example failure can be seen here or here.

[ RUN      ] IpVersionsClientType/OnDemandVhdsIntegrationTest.MultipleUpdates/IPv4_EnvoyGrpc_Unified_Rds
[2026-08-31 15:58:17.696][59507][critical][assert] [test/integration/http_integration.cc:574] assert failure: 0. Details: Timed out waiting for new connection.
[2026-08-31 15:58:17.696][59507][error][envoy_bug] [./source/common/common/assert.h:60] stacktrace for envoy bug
[2026-08-31 15:58:17.735][59507][error][envoy_bug] [./source/common/common/assert.h:64] #0 Envoy::HttpIntegrationTest::waitForNextUpstreamConnection() [0x56186bb8a1a9]
[2026-08-31 15:58:17.756][59507][error][envoy_bug] [./source/common/common/assert.h:64] #1 Envoy::HttpIntegrationTest::waitForNextUpstreamRequest() [0x56186bb88308]
[2026-08-31 15:58:17.782][59507][error][envoy_bug] [./source/common/common/assert.h:64] #2 Envoy::HttpIntegrationTest::waitForNextUpstreamRequest() [0x56186bb8ab31]
[2026-08-31 15:58:17.797][59507][error][envoy_bug] [./source/common/common/assert.h:64] #3 Envoy::(anonymous namespace)::OnDemandVhdsIntegrationTest_MultipleUpdates_Test::TestBody() [0x56186ba2b865]
[2026-08-31 15:58:17.814][59507][error][envoy_bug] [./source/common/common/assert.h:64] #4 testing::internal::HandleSehExceptionsInMethodIfSupported<>() [0x56186f366344]
[2026-08-31 15:58:17.829][59507][error][envoy_bug] [./source/common/common/assert.h:64] #5 testing::internal::HandleExceptionsInMethodIfSupported<>() [0x56186f34af86]
[2026-08-31 15:58:17.849][59507][error][envoy_bug] [./source/common/common/assert.h:64] #6 testing::Test::Run() [0x56186f332347]
[2026-08-31 15:58:17.868][59507][error][envoy_bug] [./source/common/common/assert.h:64] #7 testing::TestInfo::Run() [0x56186f332c31]
[2026-08-31 15:58:17.884][59507][error][envoy_bug] [./source/common/common/assert.h:64] #8 testing::TestSuite::Run() [0x56186f3333ca]
[2026-08-31 15:58:17.900][59507][error][envoy_bug] [./source/common/common/assert.h:64] #9 testing::internal::UnitTestImpl::RunAllTests() [0x56186f340f53]
[2026-08-31 15:58:17.915][59507][error][envoy_bug] [./source/common/common/assert.h:64] #10 testing::internal::HandleSehExceptionsInMethodIfSupported<>() [0x56186f368d34]
[2026-08-31 15:58:17.932][59507][error][envoy_bug] [./source/common/common/assert.h:64] #11 testing::internal::HandleExceptionsInMethodIfSupported<>() [0x56186f34cfd6]
[2026-08-31 15:58:17.948][59507][error][envoy_bug] [./source/common/common/assert.h:64] #12 testing::UnitTest::Run() [0x56186f34085b]
[2026-08-31 15:58:17.965][59507][error][envoy_bug] [./source/common/common/assert.h:64] #13 RUN_ALL_TESTS() [0x56186df8b9a1]
[2026-08-31 15:58:17.982][59507][error][envoy_bug] [./source/common/common/assert.h:64] #14 Envoy::TestRunner::runTests() [0x56186df8af59]
[2026-08-31 15:58:17.998][59507][error][envoy_bug] [./source/common/common/assert.h:64] #15 main [0x56186df88e20]

The theory of this fix is from a deep AI inspection using GPT Sol. I understand what the change is doing and have reviewed it, but I wouldn't bet heavily on it actually fixing the problem.

The explanation is, cleanupUpstreamAndDownstream() previously waited only for the fake upstream’s local socket close. This did not guarantee that Envoy’s worker had received the FIN and removed the connection from its HTTP connection pool. A subsequent request could therefore briefly reuse the closed connection instead of creating a new one, causing waitForNextUpstreamConnection() to time out as the request never provokes a new connection.

For TCP upstreams, this cleanup now half-closes the fake upstream and waits for Envoy to close the reverse direction, thereby proving Envoy observed the shutdown so it's really cleaned up. It then runs a worker-thread barrier that effectively drains the events in the dispatchers, to ensure Envoy’s connection-close callbacks have removed the old pool client, before cleanup returns.

HTTP/3 still retains the previous behavior because QUIC does not support half-close, so the same race-flake might still be available if any test uses HTTP/3 upstreams with a similar sequence of events.

For some cases where half-close isn't supported, the worker-barrier is still included, which may suffice to make even those cases less flaky, given that the flakiness this targets was already around a 1/1000 case.

Risk Level: Test-only.
Testing: Yes it is.
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

[repokitteh-read-only]

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #47096 was opened by ravenblackx.

see: more, trace.

[phlax]

bot review ...

test/integration/http_integration.cc (~L515-520): the ordering guarantee is conditional on envoy.reloadable_features.connection_close_through_filter_manager. When that feature is enabled and the filter manager has pending data, filter_manager_.onConnectionClose(close_action) (connection_impl.cc L343) defers the close instead of reaching closeSocket() synchronously. Callbacks then fire in a later dispatcher iteration — but since Envoy only writes its own FIN via closeSocket(), the test's waitForDisconnect() still can't complete before the callback, so the fix remains correct; a comment noting this dependency would help future readers.

test/integration/fake_upstream.cc (~L502-518): halfCloseAndWaitForDisconnect relies on Envoy responding to the FIN with a full close. That's guaranteed only because upstream connections have enable_half_close_ false (connection_impl.cc L824-827 translates EOF into PostIoAction::Close). Any test enabling upstream half-close (e.g. tcp_proxy with idle_timeout/half-close support) would hang here — worth a comment or a guard.

[ravenblackx]

/retest

[ravenblackx]

/retest
(does this work for "cancelled"?)
(hm, bot didn't even respond with confused-face, maybe it doesn't like extra text at all?)

[ravenblackx]

/retest

[ravenblackx]

/retest

[ravenblackx]

/retest

#47167 attempts to deflake the unrelated flake that keeps failing here.

[ravenblackx]

/retest