@@ -7,24 +7,29 @@ The CUDA backend is the ExecuTorch solution for running models on NVIDIA GPUs. I
 - **Optimized GPU Execution**: Uses AOTInductor to generate highly optimized CUDA kernels for model operators
 - **Triton Kernel Support**: Leverages Triton for GEMM (General Matrix Multiply), convolution, and SDPA (Scaled Dot-Product Attention) kernels.
 - **Quantization Support**: INT4 weight quantization with tile-packed format for improved performance and reduced memory footprint
-- **Cross-Platform**: Supports both Linux and Windows platforms
+- **Cross-Platform**: Runs on Linux, and on Windows from C++. Programs for Windows are exported on Linux.
 - **Multiple Model Support**: Works with various models including LLMs, vision-language models, and audio models
 
 ## Target Requirements
 
 Below are the requirements for running a CUDA-delegated ExecuTorch model:
 
-- **Hardware**: NVIDIA GPU with CUDA compute capability
-- **CUDA Toolkit**: CUDA 11.x or later (CUDA 12.x recommended)
-- **Operating System**: Linux or Windows
-- **Drivers**: PyTorch-Compatible NVIDIA GPU drivers installed
+- **Hardware**: NVIDIA GPU with compute capability 8.0 (Ampere) or newer. This is the floor the
+  prebuilt packages are built for.
+- **CUDA**: 13 for the prebuilt packages. On Linux they bring the CUDA runtime with them. On
+  Windows it comes from a CUDA 13 toolkit install. A build from source also supports CUDA 12.6.
+- **Operating System**: Linux, or Windows from C++ for programs exported on Linux (see
+  [CUDA on Windows](../../using-executorch-cpp.md#cuda-on-windows))
+- **Drivers**: An NVIDIA driver that supports the CUDA version you use: CUDA 13 for the prebuilt
+  packages.
 
 ## Development Requirements
 
 To develop and export models using the CUDA backend:
 
-- **Python**: Python 3.8+
+- **Python**: Python 3.11 to 3.15
 - **PyTorch**: PyTorch with CUDA support
+- **CUDA Toolkit**: export compiles kernels with `nvcc`, the CUDA compiler, so it needs a CUDA toolkit
 - **ExecuTorch**: Install ExecuTorch with CUDA backend support
 
 ## Using the CUDA Backend
@@ -83,7 +88,7 @@ For a complete working example, see the [CUDA export script](https://github.com/
 
 ## Runtime Integration
 
-To run the model on device, use the standard ExecuTorch runtime APIs. See [Running on Device](getting-started.md#running-on-device) for more information.
+To run the model on device, use the standard ExecuTorch runtime APIs. See [Running on Device](../../getting-started.md#running-on-device) for more information.
 
 When building from source, pass `-DEXECUTORCH_BUILD_CUDA=ON` when configuring the CMake build to compile the CUDA backend.
 
@@ -140,13 +145,10 @@ exec_program = et_program.to_executorch(
             skip_d2h_for_method_outputs=True,
         ),
         enable_non_cpu_memory_planning=True,
-        # Required alongside the skips. Memory planning reserves a buffer for graph inputs by
-        # default, and the runtime fills a planned input by copying the caller's memory into that
-        # buffer, which reintroduces the copy you just asked to skip. On device memory that copy is
-        # a host memcpy into a device pointer, which is undefined.
-        # Outputs stay planned, so the runtime allocates them on the device and hands back a
-        # pointer to its own buffer. See the note below for when to unplan them instead.
-        memory_planning_pass=MemoryPlanningPass(alloc_graph_input=False),
+        # Keeps the method from reserving its own input and output memory.
+        memory_planning_pass=MemoryPlanningPass(
+            alloc_graph_input=False, alloc_graph_output=False
+        ),
     )
 )
 ```
@@ -157,7 +159,8 @@ outputs = method([torch.randn(4, 64, device="cuda")])
 ```
 
 Outputs also stay on the device, which is what you want when the next stage of your pipeline is
-also on the GPU.
+also on the GPU. From Python, each output comes back in GPU memory that the next call does not
+overwrite, unless you pass `clone_outputs=False`.
 
 ### Things worth knowing
 
@@ -170,24 +173,17 @@ it to `False` while asking for either skip raises a `ValueError`, because copy i
 during device-aware memory planning.
 
 **Inputs must be unplanned**, via `MemoryPlanningPass(alloc_graph_input=False)`. Without it the
-program reserves its own input buffer and the runtime copies the caller's memory into it, so the
-copy comes back at run time. On device memory that copy is a host memcpy into a device pointer,
-which is undefined, so the program crashes rather than returning a wrong answer.
-
-**Outputs have two valid shapes, and which one you want depends on the language.**
-
-Leave outputs planned, which is the default, when the caller cannot supply device memory itself.
-The planned buffer is allocated on the device because `enable_non_cpu_memory_planning` is on, and
-`skip_d2h_for_method_outputs` means nothing copies it back, so the result stays on the device and
-the caller receives a pointer to the runtime's own buffer. That buffer belongs to the method and is
-reused by the next execution, so read or copy the result before running again. This is the shape
-every Python caller wants, and it is what the exporters in this repository use, for example
-`examples/models/gemma4_31b/export.py`.
-
-Set `alloc_graph_output=False` only when the caller supplies the output memory itself, which today
-means a C++ caller handing the runtime a device tensor through `Module::set_output`. The Python
-bindings allocate host memory for an unplanned output, so a device resident method exported this
-way cannot run from Python.
+program reserves its own input buffer on the device, and the runtime refuses to copy the caller's
+tensor into it. The call fails with error `0x10` (`NotSupported`), and the log says
+`Planned-buffer copy needs host memory on both sides`.
+
+**Leave outputs unplanned too**, via `alloc_graph_output=False`. Then each result lands in memory
+you own, with no extra copy. A C++ caller must give each unplanned output a buffer once with
+`Module::set_output`, before the first run: GPU memory when the output copy is skipped, host
+memory otherwise. Pass each output's index to `set_output`; the C++ example below sets output 0. If
+you keep outputs planned instead, they live in the method's own memory and the next run overwrites
+them. Python's `execute()` then copies each output on every call, unless you pass
+`clone_outputs=False`.
 
 **Per-method selection is on the outer config, not on the flags.** Pass a dict of
 `PropagateDeviceConfig` keyed by method name:
@@ -221,6 +217,11 @@ device right after the scalar type. Nothing is copied:
 
 ```cpp
 auto input = from_blob(device_data, {rows, columns}, ScalarType::Float, DeviceType::CUDA);
+auto output = from_blob(device_result, {rows, columns}, ScalarType::Float, DeviceType::CUDA);
+// Once, before the first run. Every run then writes its result to device_result.
+if (module.set_output(output) != Error::Ok) {
+  return 1;
+}
 auto result = module.forward(input);
 ```
 
@@ -235,6 +236,7 @@ across, which costs one transfer:
 ```cpp
 auto host_input = make_tensor_ptr({rows, columns}, std::move(data));
 auto input = clone_tensor_ptr_to(host_input, DeviceType::CUDA);
+// Set the output with module.set_output first, as above.
 auto result = module.forward(input);
 ```
 
 
@@ -37,8 +37,10 @@ pip install executorch torch \
 | NVIDIA GPU, CUDA 13.2 | `cu132` |
 | NVIDIA GPU, CUDA 13.4 | `cu134` |
 
-The CUDA packages are built for Linux, on x86_64 and ARM64. Use the `cpu`
-variant on macOS and on Windows. If your CUDA version is not in the table,
+The CUDA packages are built for Linux, on x86_64 and ARM64, and for Windows on
+x86_64. On Windows they run CUDA programs from C++, and programs for the CUDA
+backend are exported on Linux (see [CUDA on Windows](using-executorch-cpp.md#cuda-on-windows)).
+Use the `cpu` variant on macOS. If your CUDA version is not in the table,
 choose the closest lower one with the same major version, because CUDA works
 across minor versions but not across major ones. There is no package for CUDA 12,
 so on CUDA 12 use the `cpu` variant or build from source. On CUDA 13.0 or 13.1 no
@@ -54,8 +56,7 @@ To get a change that has landed on the `main` branch but is not in a release
 yet, use a nightly build. These are rebuilt every day. Put `nightly/` in front
 of the variant name, for example `nightly/cu132`, and add `--pre` to the
 command, otherwise pip skips development versions. Nightly builds cover the same
-variants. CUDA 13.4 is the newest, and until the next release it is in nightly
-builds only.
+variants.
 
 The second index is needed because a bare `--index-url` replaces PyPI instead of
 adding to it, and some dependencies are published only on PyPI.
 
@@ -27,9 +27,9 @@
 class PropagateDeviceConfig:
     """Controls whether the runtime copies method inputs and outputs across the device boundary.
 
-    Skipping a copy also means the runtime must not reserve its own buffer for the tensor, or it
-    would fill that buffer from the caller's memory and the copy would come back. Memory planning
-    allocates graph inputs and outputs by default, so a program using either skip below needs::
+    Skipping a copy also means memory planning should not reserve its own buffer for the tensor.
+    Memory planning allocates graph inputs and outputs by default, so a program using both skips
+    below should use::
 
         ExecutorchBackendConfig(
             propagate_device_config=PropagateDeviceConfig(
@@ -43,7 +43,9 @@ class PropagateDeviceConfig:
         )
 
     Without ``alloc_graph_input=False`` the runtime reserves device memory for an input the caller
-    already owns, then copies into it with a host memcpy, which is undefined for device memory.
+    already owns and refuses to copy into it, so every call fails with ``NotSupported``. Planned
+    outputs work, but they live in the method's memory, which the next run overwrites, and the
+    Python ``execute()`` copies them unless it gets ``clone_outputs=False``.
     """
 
     # When True, method-level input tensors that feed directly into a device
@@ -52,17 +54,21 @@ class PropagateDeviceConfig:
     # pre-staged on GPU.
     #
     # Pair with MemoryPlanningPass(alloc_graph_input=False), or the runtime reserves its own buffer
-    # for the input and copies the caller's memory into it, which is what this exists to avoid.
-    # A dict can be used to set per-method values, keyed by method name.
+    # for the input and refuses the caller's tensor.
+    # For per-method values, pass a dict of PropagateDeviceConfig, keyed by method name, as the
+    # propagate_device_config of ExecutorchBackendConfig.
     skip_h2d_for_method_inputs: Union[bool, Dict[str, bool]] = False
 
     # When True, device delegate outputs that are directly method outputs
     # are NOT wrapped with _d2h_copy. The method outputs stay on device.
     # Useful for cross-method GPU pipelines where the next method consumes
     # GPU tensors directly.
     #
-    # Pair with MemoryPlanningPass(alloc_graph_output=False) for the same reason as the input flag.
-    # A dict can be used to set per-method values, keyed by method name.
+    # Planned outputs work, but they live in the method's memory, which the next run overwrites,
+    # and the Python runtime copies them. MemoryPlanningPass(alloc_graph_output=False) avoids
+    # that copy, and then a C++ caller must set each output with Module::set_output.
+    # For per-method values, pass a dict of PropagateDeviceConfig, keyed by method name, as the
+    # propagate_device_config of ExecutorchBackendConfig.
     skip_d2h_for_method_outputs: Union[bool, Dict[str, bool]] = False
 
     def __hash__(self) -> int: