Skip to content

APM: initial FilterManager implementation without offloaded field streaming support - #47119

Open
penguingao wants to merge 7 commits into
envoyproxy:mainfrom
penguingao:apm_filter_manager_take_3_jetski
Open

APM: initial FilterManager implementation without offloaded field streaming support#47119
penguingao wants to merge 7 commits into
envoyproxy:mainfrom
penguingao:apm_filter_manager_take_3_jetski

Conversation

@penguingao

@penguingao penguingao commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Commit Message:
Initial implementation of APM's FilterManager that coordinates sending the parsed JSON object through AI filters. This version only supports streaming the JSON object itself, without the ability to stream the offloaded payload through the filter chain yet. The offloaded payload streaming is a TODO.

We also add re-serialization, so that a JSON object can be normalized / transcoded by the filter chain.

There is desire to run two instances of APM in legacy deployments, where one APM runs at early point of downstream filter chain, one runs on the upstream filter chain. The idea is to be able to perform other normal HTTP functionalities after APM, and once the routing decision is made, an upstream filter performs transcoding to the final model provider. To support this, and help filters between the two APMs to access the parsed JSON, we provide a JsonWithExtBuf that correctly indexes the payload that goes through the filters. This index is also useful for the later APM to avoid another pass of parsing.

This is part of #44681

Additional Description: https://github.com/penguingao/thoughts/blob/main/20260727_AI_FILTER.md
Risk Level: medium
Testing: unit tests
Docs Changes: n/a WIP filter
Release Notes: n/a WIP filter
Platform Specific Features: no

Signed-off-by: Peng Gao <pengg@google.com>
@repokitteh-read-only

Copy link
Copy Markdown

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #47119 was opened by penguingao.

see: more, trace.

Signed-off-by: Peng Gao <pengg@google.com>
Signed-off-by: Peng Gao <pengg@google.com>
Signed-off-by: Peng Gao <pengg@google.com>
@penguingao
penguingao marked this pull request as ready for review September 1, 2026 15:33
Signed-off-by: Peng Gao <pengg@google.com>
Signed-off-by: Peng Gao <pengg@google.com>
…er manager

Signed-off-by: Peng Gao <pengg@google.com>
@penguingao

Copy link
Copy Markdown
Contributor Author

/assign @botengyao @tyxia @wbpcode

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants