Skip to content

Commit ff4cf27

Browse files
authored
Let TensorPtrMaker carry the device its buffer lives on (pytorch#21763)
TensorPtrMaker could express every property of a tensor except where its data resides. make_tensor_ptr already takes a device as its last parameter, defaulting to CPU, and the builder never passed one, so for_blob(...).make_tensor_ptr() always produced a CPU labelled tensor. A caller wrapping accelerator memory therefore got a tensor claiming to be on the host. Nothing rejects that at the boundary, so the label is simply wrong, and whatever reads it next acts on the wrong answer. For memory the host cannot address, that means the host follows a pointer it must not. The setter is unqualified, matching the five setters already on this builder, so a caller can pick a device at run time and apply it to a named builder before finishing it. Test Plan: extension/tensor/test/tensor_ptr_maker_test.cpp gains four cases. The file had no device coverage at all, which is why the missing setter went unnoticed. ForBlobDefaultsToCPU the default is still CPU, so existing callers are unaffected. Type only: the default index is 0 in one build mode and -1 in the other ForBlobCarriesTheDeviceItIsGiven an explicitly set device type and index reach the tensor ForBlobDeviceSurvivesLaterSetters a setter running after it does not drop it, asserted with a non contiguous stride so the case fails rather than passing for an unrelated reason ForBlobDeviceAcceptsANamedBuilder the run time choice above, which an rvalue only qualifier would have rejected Compiled and ran all four against the shipped headers on Linux x86_64. The wheel build sets EXECUTORCH_BUILD_TESTS=OFF, so these do not run there; they run in the unit test build. Three of the four do not compile without this change, no member named 'device', so they are not vacuous. The default case is the exception: it only reads the accessor, which already existed, and would compile against the old builder while reporting CPU.
1 parent 1ebd320 commit ff4cf27

2 files changed

Lines changed: 71 additions & 1 deletion

File tree

‎extension/tensor/tensor_ptr_maker.h‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,22 @@ class TensorPtrMaker final {
9292
return std::move(*this);
9393
}
9494

95+
/**
96+
* Labels the tensor with the device where its buffer already lives.
97+
*
98+
* No data is moved, matching the corresponding `make_tensor_ptr` parameter.
99+
*
100+
* @param device The device on which the data buffer resides. Give the index
101+
* explicitly: a device constructed without one is index 0 in this build and
102+
* -1 in the ATen build, where it then compares unequal to the same device
103+
* carrying an explicit index.
104+
* @return Rvalue to this TensorPtrMaker for method chaining.
105+
*/
106+
TensorPtrMaker&& device(executorch::aten::Device device) {
107+
device_ = device;
108+
return std::move(*this);
109+
}
110+
95111
/**
96112
* Creates and returns a TensorPtr instance using the properties set in this
97113
* TensorPtrMaker.
@@ -106,7 +122,8 @@ class TensorPtrMaker final {
106122
std::move(strides_),
107123
type_,
108124
dynamism_,
109-
std::move(deleter_));
125+
std::move(deleter_),
126+
device_);
110127
}
111128

112129
/**
@@ -141,6 +158,8 @@ class TensorPtrMaker final {
141158
executorch::aten::ScalarType type_ = executorch::aten::ScalarType::Float;
142159
executorch::aten::TensorShapeDynamism dynamism_ =
143160
executorch::aten::TensorShapeDynamism::DYNAMIC_BOUND;
161+
executorch::aten::Device device_ =
162+
executorch::aten::Device(executorch::aten::DeviceType::CPU);
144163
};
145164

146165
/**

‎extension/tensor/test/tensor_ptr_maker_test.cpp‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -506,3 +506,54 @@ TEST_F(TensorPtrMakerTest, CreateRandnTensorWithIntType) {
506506
EXPECT_EQ(val, 0);
507507
}
508508
}
509+
510+
TEST_F(TensorPtrMakerTest, ForBlobDefaultsToCPU) {
511+
float data[8] = {};
512+
auto tensor = for_blob(data, {2, 4}, executorch::aten::ScalarType::Float)
513+
.make_tensor_ptr();
514+
515+
// Type only: the default index is 0 in one build mode and -1 in the other, so
516+
// asserting it would pass in one and fail in the other.
517+
EXPECT_EQ(tensor->device().type(), executorch::aten::DeviceType::CPU);
518+
}
519+
520+
TEST_F(TensorPtrMakerTest, ForBlobCarriesTheDeviceItIsGiven) {
521+
float data[8] = {};
522+
auto tensor = for_blob(data, {2, 4}, executorch::aten::ScalarType::Float)
523+
.device(executorch::aten::Device(
524+
executorch::aten::DeviceType::CUDA, 3))
525+
.make_tensor_ptr();
526+
527+
EXPECT_EQ(tensor->device().type(), executorch::aten::DeviceType::CUDA);
528+
EXPECT_EQ(tensor->device().index(), 3);
529+
}
530+
531+
TEST_F(TensorPtrMakerTest, ForBlobDeviceSurvivesLaterSetters) {
532+
float data[8] = {};
533+
auto tensor = for_blob(data, {2, 4}, executorch::aten::ScalarType::Float)
534+
.device(executorch::aten::Device(
535+
executorch::aten::DeviceType::CUDA, 0))
536+
.dim_order({1, 0})
537+
.strides({1, 2})
538+
.make_tensor_ptr();
539+
540+
// The strides are asserted with a value a contiguous tensor would not have,
541+
// so a setter that silently ignored its argument would fail here rather than
542+
// pass.
543+
EXPECT_EQ(tensor->device().type(), executorch::aten::DeviceType::CUDA);
544+
EXPECT_EQ(tensor->device().index(), 0);
545+
EXPECT_EQ(tensor->strides()[0], 1);
546+
EXPECT_EQ(tensor->strides()[1], 2);
547+
}
548+
549+
TEST_F(TensorPtrMakerTest, ForBlobDeviceAcceptsANamedBuilder) {
550+
float data[8] = {};
551+
auto maker = for_blob(data, {2, 4}, executorch::aten::ScalarType::Float);
552+
maker.device(executorch::aten::Device(executorch::aten::DeviceType::CUDA, 1));
553+
auto tensor = std::move(maker).make_tensor_ptr();
554+
555+
// Choosing a device at run time and applying it to a named builder is the
556+
// case this setter exists for, and an rvalue-only qualifier would reject it.
557+
EXPECT_EQ(tensor->device().type(), executorch::aten::DeviceType::CUDA);
558+
EXPECT_EQ(tensor->device().index(), 1);
559+
}

0 commit comments

Comments
 (0)