Skip to content

Commit 5028bb4

Browse files
authored
Report a clear error instead of crashing on a device-planned copy (pytorch#21960)
Supersedes pytorch#21704, which was stacked and did not build. Same change, rebuilt on `main` with the compile error fixed. ## What is broken When the runtime fills a memory-planned tensor it copies the caller's data into the buffer it reserved, using a plain host copy: ```cpp std::memcpy(dst_data_ptr, t_src.const_data_ptr(), t_src.nbytes()); ``` That is correct for main memory. If the destination lives on an accelerator, a host copy into it is undefined and the process dies with a segmentation fault and no message. The last thing a user sees is unrelated to the cause: ``` [cuda_backend.cpp:429] Created new CUDA stream 0x10ae920 for method Segmentation fault (core dumped) ``` Nothing points at what actually went wrong, which is a program whose activations live on a device being exported so the runtime also reserves its own buffer for them. ## The change Check the assumption the copy makes, and return an error that names the fix: ``` Planned-buffer copy needs host memory on both sides: dst device CUDA, src device CPU. Export with MemoryPlanningPass(alloc_graph_input=False) to share the caller's memory. ``` Both the ATen and portable variants of `copy_tensor_data` get the same guard, so the behaviour does not depend on which runtime is built. The message is deliberately short. The runtime logs through a 256-character buffer (`runtime/platform/log.cpp`), so a longer message is silently cut off, and the part that tells a user what to change is the part worth keeping. ## Test plan A new test builds tensors tagged CPU and CUDA and checks three cases: - host to host succeeds **and the data arrives**, - a device destination is refused with `Error::NotSupported`, - a device source is refused the same way. Each tensor owns its own storage, so the host-to-host case has a distinct destination. Deleting the copy makes that assertion fail, which is the point: a test that shares one buffer between source and destination would pass whether or not the copy happened. Verified by compiling the test against the portable headers and running the host-to-host case with the copy present and then removed: ``` copy present -> pass copy removed -> fail (4 assertions) ``` ## Known gap The guard in the ATen variant has no test coverage. The new test is registered outside the `get_aten_mode_options()` loop that the neighbouring `tensor_util_test` uses, and it names `executorch::runtime::internal::`, which is `executorch::runtime::aten::internal::` in ATen mode. The portable guard is covered; the ATen one is not. Worth fixing, and left out of this change to keep it to one concern.
1 parent 069db62 commit 5028bb4

6 files changed

Lines changed: 130 additions & 3 deletions

File tree

‎runtime/core/exec_aten/util/tensor_util_aten.cpp‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -177,6 +177,21 @@ Error copy_tensor_data(const at::Tensor& t_dst, const at::Tensor& t_src) {
177177
t_src.nbytes());
178178
// Copy the source data to the preallocated memory of the destination, which
179179
// must be the same size as the source.
180+
//
181+
// Both sides have to be host memory. Reaching here with device memory means
182+
// a program planned a buffer for a tensor that lives on an accelerator, and
183+
// a host memcpy into it is undefined. Reported because the alternative is a
184+
// crash with no message.
185+
ET_CHECK_OR_RETURN_ERROR(
186+
t_dst.device().is_cpu() && t_src.device().is_cpu(),
187+
NotSupported,
188+
// Kept under the 256-char log buffer (runtime/platform/log.cpp) so the
189+
// MemoryPlanningPass hint, which is the actionable part, is not
190+
// truncated away.
191+
"Planned-buffer copy needs host memory on both sides: dst device %s, src device %s. "
192+
"Export with MemoryPlanningPass(alloc_graph_input=False) to share the caller's memory.",
193+
c10::DeviceTypeName(t_dst.device().type()).c_str(),
194+
c10::DeviceTypeName(t_src.device().type()).c_str());
180195
std::memcpy(dst_data_ptr, t_src.const_data_ptr(), t_src.nbytes());
181196
}
182197

‎runtime/core/exec_aten/util/tensor_util_portable.cpp‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -178,6 +178,20 @@ Error copy_tensor_data(
178178
"t_dst.nbytes() %zu != t_src.nbytes(). %zu",
179179
t_dst.nbytes(),
180180
t_src.nbytes());
181+
// The copy below assumes host memory on both sides. Reaching it with device
182+
// memory means a program planned a buffer for a tensor that lives on an
183+
// accelerator, and copying into it with a host memcpy is undefined.
184+
// Reported here because the alternative is a crash with no message.
185+
ET_CHECK_OR_RETURN_ERROR(
186+
t_dst.device().is_cpu() && t_src.device().is_cpu(),
187+
NotSupported,
188+
// Kept under the 256-char log buffer (runtime/platform/log.cpp) so the
189+
// MemoryPlanningPass hint, which is the actionable part, is not
190+
// truncated away.
191+
"Planned-buffer copy needs host memory on both sides: dst device %d, src device %d. "
192+
"Export with MemoryPlanningPass(alloc_graph_input=False) to share the caller's memory.",
193+
static_cast<int>(t_dst.device().type()),
194+
static_cast<int>(t_src.device().type()));
181195
std::memcpy(
182196
t_dst.mutable_data_ptr(), t_src.const_data_ptr(), t_src.nbytes());
183197
}

‎runtime/core/exec_aten/util/test/CMakeLists.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,9 @@ set(EXECUTORCH_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/../../../../..)
2020
include(${EXECUTORCH_ROOT}/tools/cmake/Test.cmake)
2121

2222
set(_test_srcs
23-
dim_order_util_test.cpp operator_impl_example_test.cpp
24-
scalar_type_util_test.cpp tensor_shape_to_c_string_test.cpp
25-
tensor_util_test.cpp
23+
copy_tensor_data_device_test.cpp dim_order_util_test.cpp
24+
operator_impl_example_test.cpp scalar_type_util_test.cpp
25+
tensor_shape_to_c_string_test.cpp tensor_util_test.cpp
2626
)
2727

2828
et_cxx_test(runtime_core_exec_aten_util_test SOURCES ${_test_srcs} EXTRA_LIBS)
Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
/*
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
* All rights reserved.
4+
*
5+
* This source code is licensed under the BSD-style license found in the
6+
* LICENSE file in the root directory of this source tree.
7+
*/
8+
9+
#include <executorch/runtime/core/exec_aten/exec_aten.h>
10+
#include <executorch/runtime/core/exec_aten/util/tensor_util.h>
11+
#include <executorch/runtime/platform/runtime.h>
12+
13+
#include <gtest/gtest.h>
14+
15+
#include <array>
16+
#include <memory>
17+
#include <vector>
18+
19+
using executorch::runtime::Error;
20+
using executorch::runtime::etensor::DeviceType;
21+
using executorch::runtime::etensor::ScalarType;
22+
using executorch::runtime::etensor::Tensor;
23+
using executorch::runtime::etensor::TensorImpl;
24+
using executorch::runtime::internal::copy_tensor_data;
25+
26+
// Copying into a memory-planned tensor uses a host memcpy. A tensor that lives
27+
// on an accelerator cannot be filled that way, and doing it anyway crashed the
28+
// process with no message, so the runtime reports it instead.
29+
class CopyTensorDataDeviceTest : public ::testing::Test {
30+
protected:
31+
void SetUp() override {
32+
executorch::runtime::runtime_init();
33+
}
34+
35+
// Each tensor gets its own storage, so a host-to-host copy has a distinct
36+
// destination to land in and the data can be checked afterwards. Sharing one
37+
// buffer would make the copy a self-memcpy, which is undefined and which no
38+
// assertion could distinguish from the copy being skipped.
39+
Tensor make(
40+
DeviceType device,
41+
std::array<float, 4> values = {0.0f, 0.0f, 0.0f, 0.0f}) {
42+
storages_.push_back(std::make_unique<std::array<float, 4>>(values));
43+
impls_.push_back(std::make_unique<TensorImpl>(
44+
ScalarType::Float,
45+
static_cast<ssize_t>(sizes_.size()),
46+
sizes_.data(),
47+
storages_.back()->data(),
48+
dim_order_.data(),
49+
strides_.data(),
50+
executorch::runtime::TensorShapeDynamism::STATIC,
51+
device));
52+
return Tensor(impls_.back().get());
53+
}
54+
55+
std::array<executorch::aten::SizesType, 1> sizes_{4};
56+
std::array<executorch::aten::DimOrderType, 1> dim_order_{0};
57+
std::array<executorch::aten::StridesType, 1> strides_{1};
58+
std::vector<std::unique_ptr<std::array<float, 4>>> storages_;
59+
std::vector<std::unique_ptr<TensorImpl>> impls_;
60+
};
61+
62+
TEST_F(CopyTensorDataDeviceTest, HostToHostIsCopied) {
63+
Tensor destination = make(DeviceType::CPU);
64+
Tensor source = make(DeviceType::CPU, {1.0f, 2.0f, 3.0f, 4.0f});
65+
EXPECT_EQ(copy_tensor_data(destination, source), Error::Ok);
66+
// Assert the data actually moved. Checking only the returned Error would
67+
// still pass if the copy were removed entirely.
68+
const float* copied = destination.const_data_ptr<float>();
69+
ASSERT_NE(copied, nullptr);
70+
EXPECT_EQ(copied[0], 1.0f);
71+
EXPECT_EQ(copied[1], 2.0f);
72+
EXPECT_EQ(copied[2], 3.0f);
73+
EXPECT_EQ(copied[3], 4.0f);
74+
}
75+
76+
TEST_F(CopyTensorDataDeviceTest, ADeviceDestinationIsRefused) {
77+
// This is the case that used to crash: a planned buffer on an accelerator,
78+
// filled by a host memcpy.
79+
Tensor destination = make(DeviceType::CUDA);
80+
Tensor source = make(DeviceType::CPU);
81+
EXPECT_EQ(copy_tensor_data(destination, source), Error::NotSupported);
82+
}
83+
84+
TEST_F(CopyTensorDataDeviceTest, ADeviceSourceIsRefused) {
85+
Tensor destination = make(DeviceType::CPU);
86+
Tensor source = make(DeviceType::CUDA);
87+
EXPECT_EQ(copy_tensor_data(destination, source), Error::NotSupported);
88+
}

‎runtime/core/exec_aten/util/test/targets.bzl‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,15 @@ def define_common_targets():
5050
],
5151
)
5252

53+
runtime.cxx_test(
54+
name = "copy_tensor_data_device_test",
55+
srcs = ["copy_tensor_data_device_test.cpp"],
56+
deps = [
57+
"//executorch/runtime/core/exec_aten/util:tensor_util",
58+
"//executorch/runtime/core/portable_type:portable_type",
59+
],
60+
)
61+
5362
runtime.cxx_test(
5463
name = "tensor_shape_to_c_string_test",
5564
srcs = ["tensor_shape_to_c_string_test.cpp"],

‎test/utils/OSSTestConfig.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,7 @@
9797
{
9898
"directory": "runtime/core/exec_aten/util/test",
9999
"sources": [
100+
"copy_tensor_data_device_test.cpp",
100101
"dim_order_util_test.cpp",
101102
"operator_impl_example_test.cpp",
102103
"scalar_type_util_test.cpp",

0 commit comments

Comments
 (0)