[UR] Static OpenCL adapter - #21983
Conversation
61e281d to
2857569
Compare
| } | ||
| #endif | ||
| #ifdef UR_STATIC_ADAPTER_OPENCL | ||
| if (!adapter_registry.adaptersForceLoaded()) { |
| @@ -0,0 +1,120 @@ | |||
| // OpenCL function definitions for dynamic loading | |||
| // Format: OCL_FUNC(name, required) | |||
| // where 'required' is 1 for OpenCL 1.0 functions that must be present, 0 for optional | |||
There was a problem hiding this comment.
how did you determine which functions are required, and which are not?
some of these don't look right to me.
There was a problem hiding this comment.
The split was based on "OpenCL 1.0 spec membership = required, anything later = optional". However, not all the functions are used by adapter, and those which are used - are required. Now the file is reduced to functions actually used by adapter.
42ffe80 to
f002c34
Compare
| parser.add_argument( | ||
| "--static-opencl-adapter", | ||
| action="store_true", | ||
| help="Build OpenCL adapter as static library with dynamic OpenCL loading (no OpenCL dependency at link time).", | ||
| ) |
There was a problem hiding this comment.
I don't think we need to add this option. We didn't add any other backend. I think this would be a very rarely used function. And we want to enable static adapters by default anyway.
There was a problem hiding this comment.
| *pNumAdapters = liveAdapter != nullptr ? 1 : 0; |
?
what is the purpose of checking ocl::loadOCLLibrary() result here?
There was a problem hiding this comment.
Applied the suggestion
There was a problem hiding this comment.
does the namespace have to be ifdefed?
In general, I suggest limiting the number of ifdefs to the minimum required, preferring to share implementations wherever possible.
There was a problem hiding this comment.
please follow the same pattern used in level-zero adapter, with these symbols defined in ur_interface_loader.hpp. You should be able to reuse the exact same templates, just with different namespace name.
There was a problem hiding this comment.
wouldn't it be simpler to rename the struct fields?
There was a problem hiding this comment.
namespace it in both modes, and remove the ifdef.
There was a problem hiding this comment.
did you modify this file manually?
please use the existing ur_interface_loader template. It shouldn't require modifications.
There was a problem hiding this comment.
Yes, that hand-edit is gone. generate_code.py now drives the existing ur_interface_loader.{cpp,hpp}.mako templates for opencl too. Only template change was parameterizing two L0-hardcoded strings (Level Zero →
${adapter}, UR_STATIC_ADAPTER_LEVEL_ZERO → UR_STATIC_ADAPTER_${Adapter});
There was a problem hiding this comment.
why? is there a symbol conflict?
This should get resolved by creating a common library for the adapter. #21845 already does that here:
https://github.com/intel/llvm/pull/21845/changes#diff-e83af30e7502eb85108ac495d4a4d50775e6b0eba4e27e8b682c797f8b8a168fR26
There was a problem hiding this comment.
Reverted for now, should be fixed after #21845 merge
4148cd8 to
7e87593
Compare
7e87593 to
5918f7d
Compare
a3e5bbe to
cb907b8
Compare
7170f4b to
1e647d8
Compare
c755601 to
3e27cc9
Compare
554972b to
50ebd35
Compare
| // The device above is a raw reference into the platform's device list, which | ||
| // is owned by the platform. Hold a shared_ptr to the owning platform so that | ||
| // `MDevice` (and the per-device queue list it is registered in) is guaranteed | ||
| // to outlive this queue. Without this, a queue_impl kept alive by a deferred | ||
| // scheduler command can outlive its device_impl and crash in ~queue_impl when | ||
| // unregistering from the (already freed) device's queue list. | ||
| const std::shared_ptr<platform_impl> MPlatform; |
There was a problem hiding this comment.
The fix in queue_impl is not directly related to the PR's core purpose, still the issue (pre-existing, but static linking probably changed timing and the problem surfaced) prevented Windows tests to pass - "Queue not found" assert was fired on destructor of queue_impl.
You are probably right to move the fix to separate PR. I did #22481 for this.
| ur_result_t mock_urAdapterGetInfo(void *pParams) { | ||
| const auto ¶ms = *static_cast<ur_adapter_get_info_params_t *>(pParams); | ||
| if (*params.ppropName == UR_ADAPTER_INFO_BACKEND) { | ||
| if (*params.ppPropValue) { | ||
| *reinterpret_cast<ur_backend_t *>(*params.ppPropValue) = | ||
| UR_BACKEND_OPENCL; | ||
| } | ||
| if (*params.ppPropSizeRet) { | ||
| **params.ppPropSizeRet = sizeof(ur_backend_t); | ||
| } | ||
| } | ||
| return UR_RESULT_SUCCESS; | ||
| } | ||
|
|
| { | ||
| // Probe the adapter before registering: urAdapterGet triggers | ||
| // loadOCLLibrary(). If the OpenCL library is absent the global | ||
| // ocl::*_ptr function pointers stay null, and any later call through | ||
| // the DDI table (e.g. urPlatformGetInfo) would segfault. | ||
| ur_dditable_t ocl_ddi = {}; | ||
| ur::opencl::urAdapterGetDdiTables(&ocl_ddi); | ||
| ur_adapter_handle_t hAdapter = nullptr; | ||
| if (ocl_ddi.Adapter.pfnGet && | ||
| ocl_ddi.Adapter.pfnGet(1, &hAdapter, nullptr) == UR_RESULT_SUCCESS) { | ||
| ocl_ddi.Adapter.pfnRelease(hAdapter); | ||
| auto &opencl = platforms.emplace_back(nullptr); | ||
| opencl.dditable = ocl_ddi; | ||
| staticOCLRegistered = true; | ||
| } | ||
| } |
There was a problem hiding this comment.
How was this handled in the normal dynamic case? I don't think statically linking changes anything here. The OpenCL adapter should return 0 platforms, and users wouldn't call any functions on it.
|
|
||
| void enableMock() { | ||
| adaptersLoadPaths.clear(); | ||
| forceLoaded = true; |
There was a problem hiding this comment.
Please add a comment why this is added here.
|
@intel/dpcpp-devops-reviewers, @intel/bindless-images-reviewers, @intel/sycl-graphs-reviewers could you take a look please. |
mmichel11
left a comment
There was a problem hiding this comment.
Command buffer changes LGTM
| ur_adapter_is_static(${adapter} adapter_is_static) | ||
| if(adapter_is_static) | ||
| # No DLL for statically linked adapter | ||
| else() |
There was a problem hiding this comment.
we can just do
if(NOT adapter_is_static) instead of having an empty branch
|
@intel/llvm-gatekeepers Please consider merge. MacOS workflow failure appears unrelated to the changes in this PR and looks like a workflow configuration issue. |
Builds the OpenCL adapter as a static library embedded in the UR loader instead of a separate runtime-loaded .so/.dll. Controlled by UR_STATIC_ADAPTER_OPENCL (default ON for SYCL builds). The OpenCL ICD itself is still resolved dynamically at runtime, so there's no OpenCL link-time dependency.
Key changes
Dynamic ICD loading — new ocl_dynamic_lib.{cpp,hpp} loads the ICD at init: LoadLibraryExA("OpenCL.dll", LOAD_LIBRARY_SEARCH_SYSTEM32) on Windows, dlopen("libOpenCL.so.1") (→ libOpenCL.so) on Linux, resolving entry points through function pointers (core_functions.def, new ocl_functions.def).
Namespace isolation — all adapter entry points moved into namespace ur::opencl to avoid symbol collisions; the loader wires DDI tables directly via ur::opencl::urGet*ProcAddrTable.
Loader integration — under UR_STATIC_ADAPTER_OPENCL the adapter is probed/registered in-process (no adapter dlopen), and discarded cleanly when no platforms are present (ur_loader.cpp, ur_adapter_registry.hpp, loader/CMakeLists.txt).
Build/linking control — opencl/CMakeLists.txt builds the static lib, drops the OpenCL link, and zeroes UR_APIEXPORT/UR_APICALL/UR_DLLEXPORT so definitions carry no __declspec(dllimport/dllexport). BuildUnifiedRuntime.cmake forces the option on, propagates it into the Windows ExternalProject, and suppresses the now-nonexistent ur_adapter_opencl.dll in the copy/install logic.
SYCL runtime — os_util.hpp: with the adapter static, __SYCL_OCL_CALL probes libOpenCL.so.1/libOpenCL.so for clRetain*/clRelease* directly (the libur_adapter_opencl.so it used to look them up in no longer exists);