APM: initial FilterManager implementation without offloaded field streaming support - #47119
Open
penguingao wants to merge 7 commits into
Open
APM: initial FilterManager implementation without offloaded field streaming support#47119penguingao wants to merge 7 commits into
penguingao wants to merge 7 commits into
Conversation
Signed-off-by: Peng Gao <pengg@google.com>
Signed-off-by: Peng Gao <pengg@google.com>
Signed-off-by: Peng Gao <pengg@google.com>
penguingao
marked this pull request as ready for review
September 1, 2026 15:33
Signed-off-by: Peng Gao <pengg@google.com>
Signed-off-by: Peng Gao <pengg@google.com>
…er manager Signed-off-by: Peng Gao <pengg@google.com>
Contributor
Author
|
/assign @botengyao @tyxia @wbpcode |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commit Message:
Initial implementation of APM's FilterManager that coordinates sending the parsed JSON object through AI filters. This version only supports streaming the JSON object itself, without the ability to stream the offloaded payload through the filter chain yet. The offloaded payload streaming is a TODO.
We also add re-serialization, so that a JSON object can be normalized / transcoded by the filter chain.
There is desire to run two instances of APM in legacy deployments, where one APM runs at early point of downstream filter chain, one runs on the upstream filter chain. The idea is to be able to perform other normal HTTP functionalities after APM, and once the routing decision is made, an upstream filter performs transcoding to the final model provider. To support this, and help filters between the two APMs to access the parsed JSON, we provide a
JsonWithExtBufthat correctly indexes the payload that goes through the filters. This index is also useful for the later APM to avoid another pass of parsing.This is part of #44681
Additional Description: https://github.com/penguingao/thoughts/blob/main/20260727_AI_FILTER.md
Risk Level: medium
Testing: unit tests
Docs Changes: n/a WIP filter
Release Notes: n/a WIP filter
Platform Specific Features: no