Skip to content

Refactor GpuVector to wrap GpuBuffer - #7372

Open
havahol wants to merge 9 commits into
OPM:masterfrom
havahol:refactor-gpu-vector-buffer
Open

Refactor GpuVector to wrap GpuBuffer#7372
havahol wants to merge 9 commits into
OPM:masterfrom
havahol:refactor-gpu-vector-buffer

Conversation

@havahol

@havahol havahol commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Refactor GpuVector to own GPU memory via GpuBuffer

Summary

Previously, GpuVector duplicated responsibilities already handled by GpuBuffer: allocation, deallocation, size tracking, and parts of host/device transfer.

This PR refactors GpuVector to wrap a GpuBuffer member, reusing GpuBuffer for raw GPU storage while keeping vector-specific operations (cuBLAS, async copies, index-set helpers, dim instead of size) in GpuVector.

The public GpuVector API is unchanged.

Changes

GpuVector refactor

  • Replaced T* m_dataOnDevice and int m_numberOfElements with GpuBuffer<T> m_buffer.
  • Delegated equivalent functionality to GpuBuffer where possible (constructors, data(), dim(), host/device copies for Dune::BlockVector, etc.).
  • GpuBuffer stores size as size_t; cuBLAS calls now use detail::to_int() at the call site instead of storing size as int in GpuVector.
  • Added checkedSize() to validate that requested sizes fit in int during construction and resize() (cuBLAS requirement).

GpuBuffer fixes and tests

Expanded test_GpuBuffer coverage, which exposed and fixed two bugs:

  • resize: growing a buffer updated m_numberOfElements but did just re-allocate the old capacity.
  • copyToHost(BlockVector): size check compared against size() instead of dim() (total scalar count = size() * block_size = dim()).

Pointer attribute helpers

  • Renamed is_gpu_pointer.hpp to gpu_pointer_attributes.hpp.
  • Added isCPUPointer() to detect whether a pointer refers to host memory (used when validating host pointers passed to GpuBuffer constructors).

Test plan

(Suggested by the LLM that fixed all my markdown encodings - not sure if you usually specify this, or if it is standard anyway)

  • Build GPU-enabled opm-simulators
  • ctest -R 'TestGpuVector|TestGpuBuffer|test_gpu_pointer_attributes' --output-on-failure
  • Run broader gpuistl tests that use GpuVector (e.g. TestGpuJac, TestGpuDILU, test_preconditioner_factory_gpu), or simply all tests.
  • Confirm tests still pass after hipify (HIP build)

Next steps

  • Move all cudaMemcpy calls into reusable utility functions in gpuistl::detail.
  • Centralize pointer-attribute checks (e.g. host vs device) in those helpers so GpuVector, GpuBuffer, and other gpuistl types can share the same logic.
  • Extend GpuBuffer with async transfers.

@kjetilly kjetilly added the manual:irrelevant This PR is a minor fix and should not appear in the manual label Sep 9, 2026
@kjetilly

kjetilly commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

jenkins build this serial rocm hipify please

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

There are correctness/API-consistency and test hygiene issues (notably GpuVector::resize(0) behavior change vs prior semantics/PR claim, plus test debug output and missing direct includes) that should be addressed before approval.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR refactors Opm::gpuistl::GpuVector to own its GPU memory via a GpuBuffer<T> member, reusing GpuBuffer for allocation/size tracking and host/device transfers while keeping vector-specific operations (cuBLAS, async copies, index-set helpers) in GpuVector. It also renames and expands pointer-attribute helpers and substantially broadens GpuBuffer test coverage, including fixes to GpuBuffer::resize() and copyToHost(BlockVector) size validation.

Changes:

  • Refactor GpuVector to wrap GpuBuffer<T> and convert cuBLAS size arguments at call sites via detail::to_int().
  • Fix and extend GpuBuffer behavior (notably resize() growth allocation and BlockVector dim() checks) with expanded unit tests.
  • Rename is_gpu_pointer helpers to gpu_pointer_attributes and add isCPUPointer().
File summaries
File Description
tests/gpuistl/test_GpuVector.cpp Adds coverage for rejecting sizes too large for cuBLAS-int constraints.
tests/gpuistl/test_GpuBuffer.cu Major test expansion covering construction, copies, resize semantics, and bool round-trips.
tests/gpuistl/test_gpu_pointer_attributes.cpp Updates/extends tests for GPU/CPU pointer attribute helpers.
opm/simulators/linalg/gpuistl/GpuVector.hpp Switches storage to GpuBuffer<T> and updates API docs/signatures accordingly.
opm/simulators/linalg/gpuistl/GpuVector.cpp Implements the GpuBuffer-backed behavior and converts cuBLAS calls to detail::to_int(dim()).
opm/simulators/linalg/gpuistl/GpuBuffer.hpp Adds CPU-pointer validation, fixes BlockVector dim() checks, and corrects resize() growth allocation.
opm/simulators/linalg/gpuistl/gpu_smart_pointer.hpp Updates include to renamed pointer-attribute helper header.
opm/simulators/linalg/gpuistl/detail/gpu_pointer_attributes.hpp New/renamed pointer attribute helpers (isGPUPointer, isCPUPointer).
CMakeLists.txt Renames the test target from is_gpu_pointer to gpu_pointer_attributes.
CMakeLists_files.cmake Updates installed header/test source lists for the renamed helper/test file.
Review details

Suppressed comments (2)

tests/gpuistl/test_gpu_pointer_attributes.cpp:25

  • This test uses std::vector and std::make_unique but does not include / directly; relying on transitive includes can break compilation depending on toolchain and include order.
    opm/simulators/linalg/gpuistl/GpuVector.hpp:441
  • The resize() documentation states that new_size must be at least 1, but the implementation previously supported resize(0) as a way to clear/release GPU memory. If resize(0) remains supported, update this note accordingly; otherwise the PR description should call out the behavior change.
     * @brief resize changes the size of the vector, preserving existing data if new size is larger
     * @param new_size the new number of elements
     * @note \p new_size must be at least 1 and within the limits of int due to restrictions of CuBlas
     * @note If new_size is larger, existing data is preserved and new elements are uninitialized
  • Files reviewed: 10/10 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread opm/simulators/linalg/gpuistl/GpuVector.cpp
Comment thread opm/simulators/linalg/gpuistl/GpuVector.hpp
Comment thread tests/gpuistl/test_GpuVector.cpp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

manual:irrelevant This PR is a minor fix and should not appear in the manual

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants