Skip to content

Commit 7569893

Browse files
authored
TOOLS: fix in-place ReduceScatter buffer size (#1342)
## Summary - use the full source count for in-place ReduceScatter - allocate the in-place destination from the count passed to UCC ## Rationale For ReduceScatter, the generator destination count describes one rank's output shard, while the source count describes the full input. In-place operation uses the destination as the source buffer, so passing the destination shard count makes the operation smaller than the Count/Size reported by `ucc_perftest`. ## Testing - built UCC with the change - verified in-place and out-of-place collective arguments at 2, 4, and 8 ranks with host/CUDA memory and TL/UCP ring/K-nomial - ran 10,240 MPI-reference ReduceScatter cases: all passed
1 parent d718c7d commit 7569893

1 file changed

Lines changed: 3 additions & 2 deletions

File tree

tools/perf/ucc_pt_coll_reduce_scatter.cc

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -54,14 +54,15 @@ ucc_status_t ucc_pt_coll_reduce_scatter::init_args(ucc_pt_test_args_t &test_args
5454
dst_header = nullptr;
5555
if (UCC_IS_INPLACE(args)) {
5656
args.src.info.count = 0;
57-
args.dst.info.count = generator->get_dst_count();
57+
/* In-place dst contains the full source buffer. */
58+
args.dst.info.count = generator->get_src_count();
5859
} else {
5960
args.src.info.count = generator->get_src_count();
6061
args.dst.info.count = generator->get_dst_count();
6162
}
6263

6364
UCCCHECK_GOTO(ucc_pt_alloc(&dst_header,
64-
generator->get_dst_count() * dt_size,
65+
args.dst.info.count * dt_size,
6566
args.dst.info.mem_type),
6667
exit, st);
6768
args.dst.info.buffer = dst_header->addr;

0 commit comments

Comments
 (0)