Skip to content

bugfix when creating GpuSeqILU0 from GPU matrix - #7342

Open
havahol wants to merge 1 commit into
OPM:masterfrom
havahol:test_preconditioner_factory_gpu-bugfix
Open

bugfix when creating GpuSeqILU0 from GPU matrix#7342
havahol wants to merge 1 commit into
OPM:masterfrom
havahol:test_preconditioner_factory_gpu-bugfix

Conversation

@havahol

@havahol havahol commented Aug 21, 2026

Copy link
Copy Markdown

The unit test test_preconditioner_factory_gpu failed with the error

417: unknown location(0): fatal error: in "TestMatrixAdapter": memory access violation at address: 0x7fa240e21c28: invalid permissions

The direct cause was that a GpuSparseMatrix was tried created by passing device pointers instead of host pointers.
These device pointers was passed on to the GpuVector constructor GpuVector(const T* dataOnHost, const size_t numberOfElements), which (somewhat surprisingly) successfully carried out a host to device cudaMemcpy.
When the matrix constructor later tried to verify that the last element of rowIndices (the pointer to device, not host, memory) was the same as numberOfNonzeroBlocks, device memory was treated as host memory causing the above error.

The bug was related to the constructor of a GpuSeqILU0 object using a GPU matrix as input (see my diff in GpuSeqILU0.hpp).

  • Old implementation: Building GpuSparseMatrix by passing device pointers into the constructor that expected host pointers.
  • Fix: Use the GpuSparseMatrix constructor that takes GpuVectors as input instead.

Relevant question: Why didn't cudeMemcpy from host to device notice that the data it copied from was a device pointer?

My LLM claims (which sounds plausible):

Why cudaMemcpy often does not complain
cudaMemcpyHostToDevice does not mean “I will CPU-dereference src like memcpy.” It means “perform a copy; I claim the direction is host→device.”

On modern CUDA with Unified Virtual Addressing (UVA) (basically all current GPUs/toolkits you care about):

  • Host and device pointers live in one address space and are distinguishable by value.
  • The runtime can (and often does) look up pointer attributes for src and dst.
  • If both are valid CUDA-known pointers, it can still perform a correct transfer even when kind is wrong — e.g. treat it as device→device under the hood.

The proposed check in the constructor GpuVector(const T* dataOnHost, const size_t numberOfElements)
to throw an exception if dataOnHost points to device memory instead of host memory would have made debugging the failing test easier.

Such checks should be made before all cudaMemcpy calls. I will add those in a separate PR, along with some refactoring of GpuVector to hold a GpuBuffer to reduce code duplication.

@atgeirr atgeirr added the manual:irrelevant This PR is a minor fix and should not appear in the manual label Aug 21, 2026
@multitalentloes

Copy link
Copy Markdown
Member

jenkins build this hipify please

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

manual:irrelevant This PR is a minor fix and should not appear in the manual

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants