Skip to content

feat(plugins): add SaveFilesAsArtifactsPlugin, porting adk-python's file-upload offload - #1412

Merged
copybara-service[bot] merged 1 commit into
google:mainfrom
svetanis:feature/save-files-as-artifacts-plugin
Oct 27, 2026
Merged

copybara-service[bot] merged 1 commit into
google:mainfrom
svetanis:feature/save-files-as-artifacts-plugin

Conversation

@svetanis

@svetanis svetanis commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

2. Or, if no issue exists, describe the change:

Problem:

adk-python deprecated save_input_blobs_as_artifacts in favor of SaveFilesAsArtifactsPlugin.
adk-java has the parameter but not the plugin, so RunConfig.saveInputBlobsAsArtifacts(true) is the
only way to keep uploaded bytes out of every later LLM request — and it loses two things:

  1. The uploaded file name is discarded. Runner names every artifact
    artifact_{invocationId}_{index} and never reads Blob.displayName, so an upload of report.pdf
    is stored under an opaque id.
  2. Nothing is reported to the session. The user event carries a state delta only, so
    EventActions.artifactDelta stays empty and the session's artifact bookkeeping never records the
    upload.

Solution:

A new SaveFilesAsArtifactsPlugin in com.google.adk.plugins, registered like any other plugin. For
each inlineData part of the incoming user message it saves the part to the configured
BaseArtifactService — named from Blob.displayName, falling back to the existing
artifact_{invocationId}_{index} — replaces it with [Uploaded Artifact: "<name>"] in the message
that reaches the model and the session, and reports the saved versions through
EventActions.artifactDelta. When the artifact service stores the file at a URI the model can read
(gs, https or http), a FileData reference is added after the placeholder
(attachFileReference, on by default). A file over 20 MB is not saved and is replaced with an
[Upload Error: ...] note. A failed save keeps the original part and logs, without failing the
invocation. The failure is reported only through the logger, matching adk-python — so an artifact
service that is misconfigured or unreachable degrades silently to no offload, with each upload
continuing to reach the model inline.

The saved versions are kept in InvocationContext.callbackContextData() until the first agent's
beforeAgentCallback, so nothing is written to session state. On a resumed invocation, Runner builds
the agent's context separately from the one onUserMessageCallback receives, so callbackContextData
does not carry over there and the versions are not reported; that is a separate Runner bug, reported
in #1695 and fixed in #1696.

One existing source file changes: BaseAgent. For a callback that returns no content, callCallback
emitted a before/after-agent callback event only when the callback changed state, so a callback that only set an artifact delta had its versions
dropped. It now also emits when artifactDelta is non-empty, the same rule as adk-python's
BaseAgent.

The only new public surface is the plugin class itself. Registration uses the three surfaces that
already exist (Runner.Builder.plugins, App.Builder.plugins, InMemoryRunner).

File Concern
core/src/main/java/…/plugins/SaveFilesAsArtifactsPlugin.java new: the two hooks: save, name, replace, report the versions
core/src/test/java/…/plugins/SaveFilesAsArtifactsPluginTest.java new: 31 tests
core/src/main/java/…/agents/BaseAgent.java one condition in callCallback
core/src/test/java/…/agents/BaseAgentTest.java 4 new tests

Kept from the first version:

  • With no artifact service configured the hook returns Maybe.empty(), where Python returns the
    unmodified message. Same outcome for this plugin, but PluginManager early-exits on the first
    non-empty return — so Python's form also suppresses every later plugin's
    on_user_message_callback whenever this one is disabled. An empty Maybe cannot.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.
Tests run: 31, Failures: 0, Errors: 0, Skipped: 0 -- in com.google.adk.plugins.SaveFilesAsArtifactsPluginTest
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0 -- in com.google.adk.agents.BaseAgentTest

SaveFilesAsArtifactsPluginTest (31) follows adk-kotlin's test for the same plugin, without Mockito:
the placeholder and the file reference, the URI schemes, the size limit and its rounding, failed and
partly failed saves (thrown, and returned as a failed Single), a blob with no display name,
the same name twice, a message with no parts, nothing saved or reported before subscription,
versions reported once, never for a failed save and only by the plugin that saved them, and an InMemoryRunner test across two turns: the model
sees the placeholder, each turn's first agent event carries the version, the stored user event holds
the placeholder, and the stored session state is empty.

BaseAgentTest adds the artifact counterpart of the existing state-delta tests: a before or after
callback, under runAsync and runLive, that saves an artifact through CallbackContext.saveArtifact
and returns empty now produces an event carrying the version, with no state delta.

Manual End-to-End (E2E) Tests:

  • Verified end-to-end, against the offline stub and against gemini-3.5-flash.

A six-row harness builds a Runner explicitly (an InMemoryRunner cannot substitute a failing
artifact service) and registers the plugin. Below, flag means
RunConfig.saveInputBlobsAsArtifacts(true) and plugin means SaveFilesAsArtifactsPlugin.

Row Mechanism Upload Result
A flag named blob stored as artifact_<invocationId>_1, artifactDelta (none)
B plugin named blob stored as report.pdf, artifactDelta {report.pdf=0}
C plugin unnamed blob falls back to artifact_<invocationId>_1 — parity with the flag
D plugin text only passes straight through
E plugin, failing store named blob original part kept, invocation still completes
F plugin + flag named blob plugin wins; one artifact, no double save, no crash

A and B are the change, side by side:

--- Run A: RunConfig.saveInputBlobsAsArtifacts(true) ---
  artifacts stored in the session   : artifact_e-099672fd-6faf-4a01-94c5-3d218b44859b_1
  EventActions.artifactDelta        : (none)
  message appended to the session   : "Uploaded file: artifact_e-099672fd-…_1. It has been saved to the artifacts"

--- Run B: SaveFilesAsArtifactsPlugin ---
  artifacts stored in the session   : report.pdf
  EventActions.artifactDelta        : {report.pdf=0}
  message appended to the session   : "[Uploaded Artifact: "report.pdf"]"

Both offload the payload losslessly and both show the model a placeholder — the change is the file
name and the bookkeeping, not the offload.

Row F is the migration row: the plugin replaces every inlineData part it saves before the runner's
flag loop sees the message, so enabling both does not save a file twice.

A second harness covers the review changes, one row per behavior:

Row Setup Result
A two upload turns artifactDelta per turn {report.pdf=0}, {report.pdf=1}; stored session state empty
B store at gs:// URIs (stub model) placeholder plus gs://demo-bucket/report.pdf/0 reference
C store throws from saveArtifact turn completes, original part kept, log names no file
D 21 MB upload [Upload Error: File large.pdf (21.00 MB) …], nothing stored

Checklist

  • I have read the CONTRIBUTING.md document.
  • My pull request contains a single commit.
  • I have performed a self-review of my own code.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

@hemasekhar-p hemasekhar-p self-assigned this Aug 10, 2026
@hemasekhar-p

Copy link
Copy Markdown
Contributor

Hi @svetanis, thank you for your contribution! We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you.

@svetanis
svetanis force-pushed the feature/save-files-as-artifacts-plugin branch from 5bbdf57 to 8ffe210 Compare August 15, 2026 06:55
@svetanis
svetanis force-pushed the feature/save-files-as-artifacts-plugin branch from 8ffe210 to 961c7ac Compare September 2, 2026 02:16
@svetanis
svetanis force-pushed the feature/save-files-as-artifacts-plugin branch from 961c7ac to 83b25e4 Compare October 18, 2026 04:44
* would write the {@code State.REMOVED} sentinel into the event's state delta, which does not
* survive a JSON round trip.
*/
final class PendingArtifactDelta {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH: The stash leaves one key in stored session state for every upload turn. Running two turns through InMemoryRunner left two temp:save_files_as_artifacts_plugin:pending_delta:<invocationId>={} entries. The direct write mutates the Session object stored by InMemorySessionService (L340-L358), and FirestoreSessionService persists it as well (L657-L696). Please follow Kotlin and keep the versions in per-invocation data (L107-L113, L120-L128), which in Java is InvocationContext.callbackContextData().

this(DEFAULT_NAME);
}

public SaveFilesAsArtifactsPlugin(String name) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH: Please port attachFileReference as in Kotlin (L46-L51, L130-L171). No SPI change is needed. Kotlin reads the stored file's URI from saveAndReloadArtifact and its version from listVersions. Java's BaseArtifactService already provides both (L62-L73, L137-L142), and GcsArtifactService already returns a gs:// fileData from it (L266-L287). Because Python and Kotlin default it to true, adding it later would change what existing callers send to the model.

fileName,
part)
.map(version -> SavedPart.saved(placeholderFor(fileName), fileName, version))
.onErrorReturn(error -> keepOriginal(part, fileName, error));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH: onErrorReturn only catches errors signaled by the returned Single. If the artifact service throws directly from saveArtifact, the whole turn fails. In a Runner test with such a service, the turn ended in IllegalStateException instead of keeping the original part. Kotlin catches any exception around the save (L90-L100). Wrapping the call in Single.defer(() -> ...saveArtifact(...)) fixes this.


/** A failed save must not fail the invocation: the original part is passed through unchanged. */
private SavedPart keepOriginal(Part part, String fileName, Throwable error) {
logger.error("Failed to save artifact '{}'; keeping the original part.", fileName, error);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HIGH: This logs the user-chosen file name, and the logged exception can repeat it. To avoid that, Kotlin logs only the part index and the exception type (L96-L97), for example logger.error("Failed to save the file in part {} ({}); keeping the original part.", index, error.getClass().getSimpleName()).

}

/** Saves one part if it carries inline data, leaving every other part untouched. */
private Single<SavedPart> savePart(InvocationContext invocationContext, Part part, int index) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MEDIUM: Please port the 20 MB cap from Kotlin (L78-L89; Python L116-L128). An oversized file is not saved and is replaced with an [Upload Error: ...] note. Without the cap, the bytes can still reach the model here: LoadArtifactsTool, which the Javadoc recommends, puts a stored PDF or image back into the request inline (L225-L234), and a failed save also keeps the blob inline. Because an error note saves no version, return the rewritten message whenever a part changed, as Kotlin does.


@Rule public MockitoRule mockitoRule = MockitoJUnit.rule();

@Mock private InvocationContext mockInvocationContext;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MEDIUM: These tests mock both contexts, so none verifies that the delta reaches an event. In Java, the delta arrives only because drain writes state, since BaseAgent emits a callback event only on a state change (L424). Mocking also hid the leftover state keys. Please add an InMemoryRunner test like Kotlin's (L410-L454) to verify that the model sees the placeholder, the first agent event carries each version, and the stored user event holds the placeholder.

}

/** One input part after the offload attempt: either untouched, or replaced by a placeholder. */
private record SavedPart(Part part, Optional<String> fileName, int version) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOW: Kotlin handles this in one loop that builds the new parts and a map of versions (L66-L113). Using that structure would remove SavedPart (an Optional record component with a placeholder version), the single-use helpers, and the filter(Optional::isPresent).map(Optional::get) step. It would also keep the two ports easy to compare once the size cap and file reference are added.

public class SaveFilesAsArtifactsPlugin extends BasePlugin {

/** Name used when the plugin is constructed without an explicit one. Matches adk-python's. */
public static final String DEFAULT_NAME = "save_files_as_artifacts_plugin";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOW: DEFAULT_NAME adds public API that neither Kotlin nor Python exposes, and no other plugin in this package exposes its default name. Please make it private, or package-private if the tests need it.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LOW: This Javadoc runs to about ten sentences, while Kotlin's covers the same points in three (L34-L45). Please trim it and use {@link com.google.adk.tools.LoadArtifactsTool} instead of naming load_artifacts.

@damianmomotgoogle damianmomotgoogle added waiting on reporter Waiting for reaction by reporter. Failing that, maintainers will eventually closed it as stale. and removed needs review labels Oct 22, 2026
@svetanis
svetanis force-pushed the feature/save-files-as-artifacts-plugin branch from 83b25e4 to 272ea6c Compare October 25, 2026 05:49
@svetanis

Copy link
Copy Markdown
Contributor Author

@damianmomotgoogle ,

Thanks for the review. Changes per comment:

  1. Pending versions in session state: they are now kept in InvocationContext.callbackContextData() and reported from beforeAgentCallback; nothing is written to session state, and PendingArtifactDelta and the run hooks are gone. Without the state write, BaseAgent emitted no event for a callback that only set an artifact delta, so BaseAgent.callCallback now also emits when artifactDelta is non-empty, as adk-python's BaseAgent does. Covered by runAsync_modelGetsThePlaceholderAndTheAgentEventReportsTheVersion (stored session state empty after two turns) and the four BaseAgentTest cases run{Async,Live}_{before,after}CallbackSavesArtifactAndReturnsEmpty_*, which save through CallbackContext.saveArtifact.
  2. attachFileReference: ported, on by default; the constructors are (), (boolean attachFileReference) and (boolean attachFileReference, String name). The URI comes from saveAndReloadArtifact and the version from listVersions. Covered by onUserMessage_savesTheFileAndPutsAPlaceholderAndReferenceInItsPlace, onUserMessage_attachFileReferenceFalse_putsOnlyAPlaceholderAndReportsTheVersion, the URI-scheme and MIME-type cases, onUserMessage_versionCannotBeReadBack_keepsThePlaceholderAndReportsNoVersion and onUserMessage_serviceListsNoVersions_reportsNoVersion.
  3. Errors thrown from saveArtifact: the save is wrapped in Single.defer. Covered by onUserMessage_saveFails_keepsTheFile and onUserMessage_oneOfTwoSavesFails_keepsOnlyTheFileThatFailed, whose fake service throws from saveArtifact, and by onUserMessage_saveReturnsAnError_keepsTheFile for a failed Single.
  4. File name in the log: the error now logs the part index and the exception type only. The size-limit warning does not log the name either.
  5. 20 MB cap: ported; an oversized file is replaced with the [Upload Error: ...] note and not saved, and the rewritten message is returned whenever a part changed. Covered by onUserMessage_fileOverTheSizeLimit_isReplacedWithAnError, onUserMessage_fileAtTheSizeLimit_isSaved and onUserMessage_fileSizeHalfwayBetweenHundredths_roundsHalfToEven.
  6. Tests on mocks: the test class now follows the Kotlin one, without Mockito. runAsync_modelGetsThePlaceholderAndTheAgentEventReportsTheVersion runs two turns through an InMemoryRunner and checks that the model sees the placeholder, each turn's first agent event carries the version, the stored user event holds the placeholder, and the stored session state is empty.
  7. One loop: onUserMessageCallback and save now follow Kotlin's: a single pass over the parts builds the new parts and the versions map. SavedPart, the helpers Kotlin does not have and the filter(Optional::isPresent) step are gone; the loop body is one method, newParts, which concatMapSingle runs once per part. As in Kotlin, a SavedFile record carries the version and the reference, each optional, with no placeholder value.
  8. DEFAULT_NAME: private.
  9. Javadoc: trimmed to three sentences, linking LoadArtifactsTool.

On a resumed invocation, Runner builds the agent's context separately from the one onUserMessageCallback receives, so callbackContextData does not carry over there; that is reported in #1695 and fixed in #1696.

@damianmomotgoogle damianmomotgoogle added ready to pull and removed waiting on reporter Waiting for reaction by reporter. Failing that, maintainers will eventually closed it as stale. labels Oct 27, 2026
@copybara-service
copybara-service Bot merged commit 1a18907 into google:main Oct 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Port SaveFilesAsArtifactsPlugin from adk-python

3 participants