bugfix when creating GpuSeqILU0 from GPU matrix - #7342
Open
havahol wants to merge 1 commit into
Open
Conversation
Member
|
jenkins build this hipify please |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The unit test
test_preconditioner_factory_gpufailed with the errorThe direct cause was that a GpuSparseMatrix was tried created by passing device pointers instead of host pointers.
These device pointers was passed on to the
GpuVectorconstructorGpuVector(const T* dataOnHost, const size_t numberOfElements), which (somewhat surprisingly) successfully carried out a host to devicecudaMemcpy.When the matrix constructor later tried to verify that the last element of
rowIndices(the pointer to device, not host, memory) was the same asnumberOfNonzeroBlocks, device memory was treated as host memory causing the above error.The bug was related to the constructor of a
GpuSeqILU0object using a GPU matrix as input (see my diff in GpuSeqILU0.hpp).GpuSparseMatrixby passing device pointers into the constructor that expected host pointers.GpuSparseMatrixconstructor that takesGpuVectors as input instead.Relevant question: Why didn't
cudeMemcpyfrom host to device notice that the data it copied from was a device pointer?My LLM claims (which sounds plausible):
The proposed check in the constructor
GpuVector(const T* dataOnHost, const size_t numberOfElements)to throw an exception if
dataOnHostpoints to device memory instead of host memory would have made debugging the failing test easier.Such checks should be made before all
cudaMemcpycalls. I will add those in a separate PR, along with some refactoring ofGpuVectorto hold aGpuBufferto reduce code duplication.