fix: report status for Kube Gateway resources rejected without an xDS change - #11385
Open
chandler-solo wants to merge 3 commits into
Open
fix: report status for Kube Gateway resources rejected without an xDS change#11385chandler-solo wants to merge 3 commits into
chandler-solo wants to merge 3 commits into
Conversation
fix: report status for Kube Gateway resources rejected without an xDS change
# Description
A resource that Kubernetes Gateway translation rejects gets no status written at all if rejecting it
does not change the proxy config. The resource keeps an empty `statuses: {}` map indefinitely, so
users get no signal that their config was rejected.
`XdsSnapWrapper.Equals` compared only the Envoy snapshot, ignoring the reports and the plugin
registry it carries. Rejecting an invalid option leaves the last good config in place, so the
snapshot after the rejection is identical to the one before it. krt therefore saw no change, kept
the previous `XdsSnapWrapper`, and discarded the plugin registry whose `classicStatusCache` held the
newly rejected resource. The status ticker in `proxy_syncer.go` reports from the registries attached
to the snapshots in `mostXdsSnapshots`, so it never saw the resource and never wrote its status.
Change detection now also compares the proxy report.
## Code changes
- Compare `proxyWithReport.Reports.ProxyReport` with `proto.Equal` in `XdsSnapWrapper.Equals`
- Add `TestXdsSnapWrapperEquals` covering the report-only change
# Context
This surfaced as a flake in `TestK8sGatewayNoValidation/VirtualHostOptions/TestConfigureInvalidVirtualHostOptions`,
which applies an invalid `VirtualHostOption` (`bad-retries`, a retry backoff with
`baseInterval` > `maxInterval`) with `alwaysAccept: true` and asserts it goes to `Rejected`. The
assertion fails with `have matcher for namespace <install-ns> which is not found` — no status entry
at all, rather than a wrong state — which is the signature of this bug rather than of a slow status
write. Two occurrences on unrelated branches:
- https://github.com/solo-io/gloo/actions/runs/32189660295/job/95881990320 (cluster-six)
- https://github.com/solo-io/gloo/actions/runs/31626638905/job/94214693219 (cluster-six)
The test is a good fit for triggering this because it asserts the proxy config is *unchanged* after
the bad VHO is applied, which is exactly the case the old equality check discarded. It passes most
of the time only because an unrelated change often perturbs the snapshot in the same window.
## Interesting decisions
Comparing the proxy report risks reintroducing the no-op report churn fixed in #11308, so this is
deliberately narrow:
- Only `ProxyReport` is compared, not `fullReports`/`ResourceReports`. The latter is keyed by
`InputResource` pointers, which are freshly allocated per translation, so comparing it would flap
on every recompute — the hot loop we want to avoid.
- `ProxyReport` is safe to compare: `MakeReport` builds it structurally from the proxy's own
deterministic ordering, it carries no timestamps (the `LastTransitionTime` problem in #11308 was
in the Gateway API `ReportMap`, not here), and its `HttpListenerReports` is a proto map, which
`proto.Equal` compares order-independently.
## Testing steps
- `TestXdsSnapWrapperEquals` fails on `main` for the `identical snapshot, report gained an error`
case and passes with this change
- `make fmt` clean, `make analyze` reports 0 issues
- `./projects/gateway2/proxy_syncer/...`, `.../translator/plugins/virtualhostoptions/...`,
`.../reports/...` and all of `./projects/gateway2/translator/...` pass
I have not run the kube e2e suite locally, so the fix to the flake itself is reasoned from the
failure signature and the code path rather than observed. Worth confirming against
`TestK8sGatewayNoValidation` in CI, ideally more than once.
## Notes for reviewers
- The changelog's `issueLink` is a placeholder pointing at the #11308 issue because I did not have a
tracking issue for this — please repoint it before merge.
- The main thing to sanity check is the churn argument above: if `ProxyReport` turns out to be
unstable across identical translations in some listener type I did not consider, this would spin
the status ticker.
# Checklist:
- [ ] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] I have added tests that prove my fix is effective or that my feature works
Signed-off-by: David L. Chandler <david.chandler@solo.io>
Author
|
Testing at https://github.com/solo-io/gloo/actions/runs/32324181750 Haven't set the bot token secret yet. |
puertomontt
approved these changes
Aug 20, 2026
Author
Posted this on the wrong PR. Ignore. |
Signed-off-by: David L. Chandler <david.chandler@solo.io>
Author
|
/kick |
Author
|
/kick-ci |
Author
|
/skip-changelog |
puertomontt
approved these changes
Aug 21, 2026
chandler-solo
enabled auto-merge (squash)
September 1, 2026 04:14
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A resource that Kubernetes Gateway translation rejects gets no status written at all if rejecting it does not change the proxy config. The resource keeps an empty
statuses: {}map indefinitely, so users get no signal that their config was rejected.XdsSnapWrapper.Equalscompared only the Envoy snapshot, ignoring the reports and the plugin registry it carries. Rejecting an invalid option leaves the last good config in place, so the snapshot after the rejection is identical to the one before it. krt therefore saw no change, kept the previousXdsSnapWrapper, and discarded the plugin registry whoseclassicStatusCacheheld the newly rejected resource. The status ticker inproxy_syncer.goreports from the registries attached to the snapshots inmostXdsSnapshots, so it never saw the resource and never wrote its status.Change detection now also compares the proxy report.
Code changes
proxyWithReport.Reports.ProxyReportwithproto.EqualinXdsSnapWrapper.EqualsTestXdsSnapWrapperEqualscovering the report-only changeContext
This surfaced as a flake in
TestK8sGatewayNoValidation/VirtualHostOptions/TestConfigureInvalidVirtualHostOptions, which applies an invalidVirtualHostOption(bad-retries, a retry backoff withbaseInterval>maxInterval) withalwaysAccept: trueand asserts it goes toRejected. The assertion fails withhave matcher for namespace <install-ns> which is not found— no status entry at all, rather than a wrong state — which is the signature of this bug rather than of a slow status write. Two occurrences on unrelated branches:The test is a good fit for triggering this because it asserts the proxy config is unchanged after the bad VHO is applied, which is exactly the case the old equality check discarded. It passes most of the time only because an unrelated change often perturbs the snapshot in the same window.
Interesting decisions
Comparing the proxy report risks reintroducing the no-op report churn fixed in #11308, so this is deliberately narrow:
ProxyReportis compared, notfullReports/ResourceReports. The latter is keyed byInputResourcepointers, which are freshly allocated per translation, so comparing it would flap on every recompute — the hot loop we want to avoid.ProxyReportis safe to compare:MakeReportbuilds it structurally from the proxy's own deterministic ordering, it carries no timestamps (theLastTransitionTimeproblem in fix: stop no-op Gateway report churn #11308 was in the Gateway APIReportMap, not here), and itsHttpListenerReportsis a proto map, whichproto.Equalcompares order-independently.Testing steps
TestXdsSnapWrapperEqualsfails onmainfor theidentical snapshot, report gained an errorcase and passes with this changeHave not run the kube e2e suite locally, so the fix to the flake itself is reasoned from the failure signature and the code path rather than observed.
Notes for reviewers
issueLinkis fix: stop no-op Gateway report churn #11308, questionableProxyReportturns out to be unstable across identical translations in some listener type I did not consider, this would spin the status ticker.Checklist: