Repository navigation
feat(plugins): add SaveFilesAsArtifactsPlugin, porting adk-python's file-upload offload - #1412
Conversation
|
Hi @svetanis, thank you for your contribution! We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you. |
5bbdf57 to
8ffe210
Compare
8ffe210 to
961c7ac
Compare
961c7ac to
83b25e4
Compare
| * would write the {@code State.REMOVED} sentinel into the event's state delta, which does not | ||
| * survive a JSON round trip. | ||
| */ | ||
| final class PendingArtifactDelta { |
There was a problem hiding this comment.
HIGH: The stash leaves one key in stored session state for every upload turn. Running two turns through InMemoryRunner left two temp:save_files_as_artifacts_plugin:pending_delta:<invocationId>={} entries. The direct write mutates the Session object stored by InMemorySessionService (L340-L358), and FirestoreSessionService persists it as well (L657-L696). Please follow Kotlin and keep the versions in per-invocation data (L107-L113, L120-L128), which in Java is InvocationContext.callbackContextData().
| this(DEFAULT_NAME); | ||
| } | ||
|
|
||
| public SaveFilesAsArtifactsPlugin(String name) { |
There was a problem hiding this comment.
HIGH: Please port attachFileReference as in Kotlin (L46-L51, L130-L171). No SPI change is needed. Kotlin reads the stored file's URI from saveAndReloadArtifact and its version from listVersions. Java's BaseArtifactService already provides both (L62-L73, L137-L142), and GcsArtifactService already returns a gs:// fileData from it (L266-L287). Because Python and Kotlin default it to true, adding it later would change what existing callers send to the model.
| fileName, | ||
| part) | ||
| .map(version -> SavedPart.saved(placeholderFor(fileName), fileName, version)) | ||
| .onErrorReturn(error -> keepOriginal(part, fileName, error)); |
There was a problem hiding this comment.
HIGH: onErrorReturn only catches errors signaled by the returned Single. If the artifact service throws directly from saveArtifact, the whole turn fails. In a Runner test with such a service, the turn ended in IllegalStateException instead of keeping the original part. Kotlin catches any exception around the save (L90-L100). Wrapping the call in Single.defer(() -> ...saveArtifact(...)) fixes this.
|
|
||
| /** A failed save must not fail the invocation: the original part is passed through unchanged. */ | ||
| private SavedPart keepOriginal(Part part, String fileName, Throwable error) { | ||
| logger.error("Failed to save artifact '{}'; keeping the original part.", fileName, error); |
There was a problem hiding this comment.
HIGH: This logs the user-chosen file name, and the logged exception can repeat it. To avoid that, Kotlin logs only the part index and the exception type (L96-L97), for example logger.error("Failed to save the file in part {} ({}); keeping the original part.", index, error.getClass().getSimpleName()).
| } | ||
|
|
||
| /** Saves one part if it carries inline data, leaving every other part untouched. */ | ||
| private Single<SavedPart> savePart(InvocationContext invocationContext, Part part, int index) { |
There was a problem hiding this comment.
MEDIUM: Please port the 20 MB cap from Kotlin (L78-L89; Python L116-L128). An oversized file is not saved and is replaced with an [Upload Error: ...] note. Without the cap, the bytes can still reach the model here: LoadArtifactsTool, which the Javadoc recommends, puts a stored PDF or image back into the request inline (L225-L234), and a failed save also keeps the blob inline. Because an error note saves no version, return the rewritten message whenever a part changed, as Kotlin does.
|
|
||
| @Rule public MockitoRule mockitoRule = MockitoJUnit.rule(); | ||
|
|
||
| @Mock private InvocationContext mockInvocationContext; |
There was a problem hiding this comment.
MEDIUM: These tests mock both contexts, so none verifies that the delta reaches an event. In Java, the delta arrives only because drain writes state, since BaseAgent emits a callback event only on a state change (L424). Mocking also hid the leftover state keys. Please add an InMemoryRunner test like Kotlin's (L410-L454) to verify that the model sees the placeholder, the first agent event carries each version, and the stored user event holds the placeholder.
| } | ||
|
|
||
| /** One input part after the offload attempt: either untouched, or replaced by a placeholder. */ | ||
| private record SavedPart(Part part, Optional<String> fileName, int version) { |
There was a problem hiding this comment.
LOW: Kotlin handles this in one loop that builds the new parts and a map of versions (L66-L113). Using that structure would remove SavedPart (an Optional record component with a placeholder version), the single-use helpers, and the filter(Optional::isPresent).map(Optional::get) step. It would also keep the two ports easy to compare once the size cap and file reference are added.
| public class SaveFilesAsArtifactsPlugin extends BasePlugin { | ||
|
|
||
| /** Name used when the plugin is constructed without an explicit one. Matches adk-python's. */ | ||
| public static final String DEFAULT_NAME = "save_files_as_artifacts_plugin"; |
There was a problem hiding this comment.
LOW: DEFAULT_NAME adds public API that neither Kotlin nor Python exposes, and no other plugin in this package exposes its default name. Please make it private, or package-private if the tests need it.
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| /** |
There was a problem hiding this comment.
LOW: This Javadoc runs to about ten sentences, while Kotlin's covers the same points in three (L34-L45). Please trim it and use {@link com.google.adk.tools.LoadArtifactsTool} instead of naming load_artifacts.
83b25e4 to
272ea6c
Compare
|
Thanks for the review. Changes per comment:
On a resumed invocation, |
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Runnerdrops thecallbackContextDatathis pluginreports its versions through; a separate
Runnerfix.2. Or, if no issue exists, describe the change:
Problem:
adk-python deprecated
save_input_blobs_as_artifactsin favor ofSaveFilesAsArtifactsPlugin.adk-java has the parameter but not the plugin, so
RunConfig.saveInputBlobsAsArtifacts(true)is theonly way to keep uploaded bytes out of every later LLM request — and it loses two things:
Runnernames every artifactartifact_{invocationId}_{index}and never readsBlob.displayName, so an upload ofreport.pdfis stored under an opaque id.
EventActions.artifactDeltastays empty and the session's artifact bookkeeping never records theupload.
Solution:
A new
SaveFilesAsArtifactsPluginincom.google.adk.plugins, registered like any other plugin. Foreach
inlineDatapart of the incoming user message it saves the part to the configuredBaseArtifactService— named fromBlob.displayName, falling back to the existingartifact_{invocationId}_{index}— replaces it with[Uploaded Artifact: "<name>"]in the messagethat reaches the model and the session, and reports the saved versions through
EventActions.artifactDelta. When the artifact service stores the file at a URI the model can read(
gs,httpsorhttp), aFileDatareference is added after the placeholder(
attachFileReference, on by default). A file over 20 MB is not saved and is replaced with an[Upload Error: ...]note. A failed save keeps the original part and logs, without failing theinvocation. The failure is reported only through the logger, matching adk-python — so an artifact
service that is misconfigured or unreachable degrades silently to no offload, with each upload
continuing to reach the model inline.
The saved versions are kept in
InvocationContext.callbackContextData()until the first agent'sbeforeAgentCallback, so nothing is written to session state. On a resumed invocation,Runnerbuildsthe agent's context separately from the one
onUserMessageCallbackreceives, socallbackContextDatadoes not carry over there and the versions are not reported; that is a separate
Runnerbug, reportedin #1695 and fixed in #1696.
One existing source file changes:
BaseAgent. For a callback that returns no content,callCallbackemitted a before/after-agent callback event only when the callback changed state, so a callback that only set an artifact delta had its versions
dropped. It now also emits when
artifactDeltais non-empty, the same rule as adk-python'sBaseAgent.The only new public surface is the plugin class itself. Registration uses the three surfaces that
already exist (
Runner.Builder.plugins,App.Builder.plugins,InMemoryRunner).core/src/main/java/…/plugins/SaveFilesAsArtifactsPlugin.javacore/src/test/java/…/plugins/SaveFilesAsArtifactsPluginTest.javacore/src/main/java/…/agents/BaseAgent.javacallCallbackcore/src/test/java/…/agents/BaseAgentTest.javaKept from the first version:
Maybe.empty(), where Python returns theunmodified message. Same outcome for this plugin, but
PluginManagerearly-exits on the firstnon-empty return — so Python's form also suppresses every later plugin's
on_user_message_callbackwhenever this one is disabled. An emptyMaybecannot.Testing Plan
Unit Tests:
SaveFilesAsArtifactsPluginTest(31) follows adk-kotlin's test for the same plugin, without Mockito:the placeholder and the file reference, the URI schemes, the size limit and its rounding, failed and
partly failed saves (thrown, and returned as a failed
Single), a blob with no display name,the same name twice, a message with no parts, nothing saved or reported before subscription,
versions reported once, never for a failed save and only by the plugin that saved them, and an
InMemoryRunnertest across two turns: the modelsees the placeholder, each turn's first agent event carries the version, the stored user event holds
the placeholder, and the stored session state is empty.
BaseAgentTestadds the artifact counterpart of the existing state-delta tests: a before or aftercallback, under
runAsyncandrunLive, that saves an artifact throughCallbackContext.saveArtifactand returns empty now produces an event carrying the version, with no state delta.
Manual End-to-End (E2E) Tests:
gemini-3.5-flash.A six-row harness builds a
Runnerexplicitly (anInMemoryRunnercannot substitute a failingartifact service) and registers the plugin. Below, flag means
RunConfig.saveInputBlobsAsArtifacts(true)and plugin meansSaveFilesAsArtifactsPlugin.artifact_<invocationId>_1,artifactDelta(none)report.pdf,artifactDelta{report.pdf=0}artifact_<invocationId>_1— parity with the flagA and B are the change, side by side:
Both offload the payload losslessly and both show the model a placeholder — the change is the file
name and the bookkeeping, not the offload.
Row F is the migration row: the plugin replaces every
inlineDatapart it saves before the runner'sflag loop sees the message, so enabling both does not save a file twice.
A second harness covers the review changes, one row per behavior:
artifactDeltaper turn{report.pdf=0},{report.pdf=1}; stored session state emptygs://URIs (stub model)gs://demo-bucket/report.pdf/0referencesaveArtifact[Upload Error: File large.pdf (21.00 MB) …], nothing storedChecklist