fix(llm): snapshot vision encoder output before caching (#1229)

## Description

In any multimodal conversation with more than one image, the model
starts describing earlier images as the most recently sent one on later
turns.

`VisionEncoder::encode` caches the `EValue` returned by
`vision_encoder.execute()` per image path. That tensor aliases the
method's reusable output buffer, so the next `execute()` (the second
image, or any later encode) overwrites the bytes behind every cached
entry. On re-prefilled turns the prefiller then splices the latest
image's embeddings into every image slot. The audio path already
snapshots its encoder output for exactly this reason (see the
`AudioSlot` comment in `multimodal_prefiller.cpp`); vision never got the
same treatment.

The fix copies the encoder output into bytes owned by the cache entry
immediately after `execute()` and serves cache hits from a tensor
wrapping those owned bytes (`unordered_map` nodes are pointer-stable, so
the blob stays valid).

The bug is backend-independent (the cache sits above the delegate), so
XNNPACK/Vulkan multimodal models are affected the same way.

### Introduces a breaking change?

- [ ] Yes
- [x] No

### Type of change

- [x] Bug fix (change which fixes an issue)
- [ ] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)

### Tested on

- [x] iOS
- [ ] Android

### Testing instructions

1. Run the example LLM app with a multimodal model (e.g. Gemma 4 E2B
multimodal) on the Multimodal LLM screen.
2. Send image A with "What's in this picture?" — answer is correct.
3. Send image B (different content) with the same question — answer is
correct.
4. Ask "What was in the FIRST picture I sent?".

Before this fix, step 4 describes image B's content (both image slots
receive B's embeddings on the re-prefilled turn). After the fix, the
model correctly recalls image A.

### Screenshots

N/A

### Related issues

N/A

### Checklist

- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [x] My changes generate no new warnings

### Additional notes

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: NorbertKlockiewicz

packages/react-native-executorch/common/runner/encoders/vision_encoder.cpp

@@ -112,7 +112,7 @@ Result<EValue> VisionEncoder::encode(const MultimodalInput &input) {
 
   auto it = embedding_cache_.find(path);
   if (it != embedding_cache_.end()) {
-    return it->second;
+    return EValue(*it->second.tensor);
   }
 
   auto shape = ET_UNWRAP(getInputShape());
@@ -128,9 +128,18 @@ Result<EValue> VisionEncoder::encode(const MultimodalInput &input) {
       chw.data(), sizes, ::executorch::aten::ScalarType::Float);
 
   auto result = ET_UNWRAP(module_->execute(kVisionEncoderMethod, image_tensor));
-  auto embedding = result[0];
-  embedding_cache_.emplace(path, embedding);
-  return embedding;
+  auto out_tensor = result[0].toTensor();
+
+  CachedEmbedding cached;
+  cached.bytes.resize(out_tensor.nbytes());
+  std::memcpy(cached.bytes.data(), out_tensor.const_data_ptr(),
+              out_tensor.nbytes());
+  cached.sizes.assign(out_tensor.sizes().begin(), out_tensor.sizes().end());
+  cached.dtype = out_tensor.scalar_type();
+  auto [entry, inserted] = embedding_cache_.emplace(path, std::move(cached));
+  entry->second.tensor = ::executorch::extension::from_blob(
+      entry->second.bytes.data(), entry->second.sizes, entry->second.dtype);
+  return EValue(*entry->second.tensor);
 }
 
 } // namespace executorch::extension::llm

packages/react-native-executorch/common/runner/encoders/vision_encoder.h

@@ -2,11 +2,14 @@
 #pragma once
 
 #include "iencoder.h"
+#include <cstdint>
 #include <executorch/extension/module/module.h>
+#include <executorch/extension/tensor/tensor.h>
 #include <executorch/runtime/core/evalue.h>
 #include <runner/multimodal_input.h>
 #include <string>
 #include <unordered_map>
+#include <vector>
 
 namespace executorch::extension::llm {
 
@@ -26,13 +29,23 @@ class VisionEncoder : public IEncoder {
     bool with_batch;
   };
 
+  // The method's output EValue aliases the runtime's reusable output buffer,
+  // which the NEXT vision_encoder.execute() overwrites — caching it directly
+  // silently turns earlier images into the most recently encoded one. Cache
+  // an owned byte snapshot instead and hand out a tensor over those bytes.
+  struct CachedEmbedding {
+    std::vector<uint8_t> bytes;
+    std::vector<::executorch::aten::SizesType> sizes;
+    ::executorch::aten::ScalarType dtype;
+    ::executorch::extension::TensorPtr tensor;
+  };
+
   ::executorch::runtime::Result<ImageShape> getInputShape() const;
   std::vector<float> preprocessImage(const std::string &path,
                                      const ImageShape &targetShape) const;
 
   ::executorch::extension::Module *module_;
-  std::unordered_map<std::string, ::executorch::runtime::EValue>
-      embedding_cache_;
+  std::unordered_map<std::string, CachedEmbedding> embedding_cache_;
 };
 
 } // namespace executorch::extension::llm