SYCL not finding my CPU

1

Hello! I would like to use some experimental SYCL features so I downloaded a recent nightly (sycl_linux.tar.gz) from Releases · intel/llvm · GitHub (tried with a few recent ones).

I would like to run on the CPU, but it is never found via LD_LIBRARY_PATH=/path-to/sycl/lib /path-to/sycl/bin/sycl-ls. More precisely I see only:

[opencl:gpu][opencl:0] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics 730 [0x4c8b] OpenCL 3.0 NEO  [22.28.23726.1]

More verbosely (LD_LIBRARY_PATH=/path-to/sycl/lib SYCL_PI_TRACE=-1 /path-to/sycl/bin/sycl-ls 1>/dev/null), my output is:

LD_LIBRARY_PATH=/path-to/sycl/lib /path-to/sycl/bin/sycl-ls
SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_level_zero.so) failed with </path-to/sycl/lib/libpi_level_zero.so: undefined symbol: zeImageViewCreateExt>
SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_cuda.so) failed with <libcuda.so.1: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_hip.so) failed with <libamdhip64.so.6: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_native_cpu.so) failed with </path-to/sycl/lib/libpi_native_cpu.so: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 15.51.1 ]
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_level_zero.so
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_cuda.so
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_hip.so
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_unified_runtime.so [ PluginVersion: 15.51.1 ]
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_native_cpu.so

I am speculating it has to do with libpi_level_zero, as I installed it with sudo apt-get install intel-level-zero-gpu level-zero (as per Level Zero — Intel® software for general purpose GPU capabilities documentation). Previously I was seeing:

SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_level_zero.so) failed with <libze_loader.so.1: cannot open shared object file: No such file or directory>

instead of

LD_LIBRARY_PATH=/path-to/sycl/lib /path-to/sycl/bin/sycl-ls SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_level_zero.so) failed with </path-to/sycl/lib/libpi_level_zero.so: undefined symbol: zeImageViewCreateExt>

And everything else was the same.

After installing level zero, I tried trivial things as (sudo) ldconfig, apt update, apt upgrade but nothing made a difference.

Orthogonally, I have the intel toolkits and I can sycl-ls which would correctly display:

[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2023.16.6.0.22_223734]
[opencl:cpu:1] Intel(R) OpenCL, 11th Gen Intel(R) Core(TM) i5-11400 @ 2.60GHz 3.0 [2023.16.6.0.22_223734]
[opencl:gpu:2] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics 730 [0x4c8b] 3.0 [22.28.23726.1]
[ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) UHD Graphics 730 [0x4c8b] 1.3 [1.3.23726]

And then being more specific:

SYCL_PI_TRACE=-1 sycl-ls 1>/dev/null

SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_opencl.so [ PluginVersion: 12.27.1 ]
SYCL_PI_TRACE[basic]: Plugin found and successfully loaded: libpi_level_zero.so [ PluginVersion: 12.27.1 ]
SYCL_PI_TRACE[-1]: dlopen(/opt/intel/oneapi/compiler/2023.2.0/linux/lib/libpi_cuda.so) failed with </opt/intel/oneapi/compiler/2023.2.0/linux/lib/libpi_cuda.so: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_cuda.so
SYCL_PI_TRACE[-1]: dlopen(/opt/intel/oneapi/compiler/2023.2.0/linux/lib/libpi_hip.so) failed with </opt/intel/oneapi/compiler/2023.2.0/linux/lib/libpi_hip.so: cannot open shared object file: No such file or directory>
SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_hip.so

There are still some (I believe) expected errors, as I have no cuda or hip backend, but the level zero is successfully found in the intel tools case. So I believe issue has to do with it.

In any case, how can I teach SYCL from nightly recognize my CPU (and my other devices)?

2

Hi @ivan,
the Intel OpenCL CPU Runtime is not part of the open-source DPC++ project. It’s a separate proprietary product that is distributed by Intel in two ways - one way is part of the oneAPI base toolkit, another way is a standalone package intel-oneapi-runtime-dpcpp-sycl-opencl-cpu in some Linux distributions.

https://www.intel.com/content/www/us/en/developer/articles/technical/intel-cpu-runtime-for-opencl-applications-with-sycl-support.html

Specifically, you need the library libintelocl.so distributed in these packages. If you have the oneAPI base toolkit already installed, you should be able to use the CPU backend with the open-source DPC++ runtime by allowing (or forcing) the loading of this library. By default, the one from oneAPI base toolkit can be found in

/opt/intel/oneapi/compiler/2024.1/lib/libintelocl.so

for the 2024.1 version. The standalone package intel-oneapi-runtime-dpcpp-sycl-opencl-cpu installs the library in

/opt/intel/oneapi/lib/intel64/libintelocl.so

You can try e.g.

LD_PRELOAD=/opt/intel/oneapi/compiler/2024.1/lib/libintelocl.so sycl-ls

to check if that works with your open-source DPC++ installation.

3

Thanks for the prompt reply @rbielski !

I installed intel-oneapi-runtime-dpcpp-sycl-opencl-cpu, but then all my devices are no longer recognized from the open source version SYCL. In addition, when I set SYCL_PI_TRACE I now see an additional message:

SYCL_PI_TRACE[-1]: dlopen(/path-to/sycl/lib/libpi_opencl.so) failed with <libsvml.so: cannot open shared object file: No such file or directory>

and this time OpenCL is not loaded: SYCL_PI_TRACE[all]: Check if plugin is present. Failed to load plugin: libpi_opencl.so.

I then attempted preloading it with: LD_PRELOAD=/opt/intel/oneapi/compiler/2023.2.0/linux/lib/libpi_opencl.so (that is my old intel oneAPI toolkit) → got:

/path-to/sycl/bin/sycl-ls: error while loading shared libraries: libsvml.so: cannot open shared object file: No such file or directory

I thought, it had to do with old toolkit version, so I just installed latest oneAPI toolkit (and so preload as LD_PRELOAD=/opt/intel/oneapi/compiler/2024.1/lib/libintelocl.so) → got:

/path-to/sycl/bin/sycl-ls: error while loading shared libraries: libtbb.so.12: cannot open shared object file: No such file or directory

On the other hand, still the intel’s sycl-ls provides the expected output. In case it helps: previously I was seeing my GPU twice, once from opencl, once from level zero - but now I see it only once from opencl, e.g. the line from the question [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) UHD Graphics 730 [0x4c8b] 1.3 [1.3.23726] is gone.

4

Hi @ivan,
I tried to reproduce the setup in a clean installation of Ubuntu 22.04 (in docker). It looks to me like both these packages don’t declare their dependencies correctly and you need to install another package to get them working. I didn’t manage to find a reasonable setup with the non-toolkit package, so I’m going to recommend you use packages which are part of the oneAPI Base Toolkit. I also found you should install the parent package intel-oneapi-runtime-opencl rather than the one I listed directly, and you need intel-oneapi-compiler-dpcpp-cpp-runtime in addition.

Here’s the steps that worked for me from scratch:

# 1. System prerequisites
apt update && apt install -y wget gpg-agent 

# 2. Register the intel repository
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \
| gpg --dearmor | tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" \
| tee /etc/apt/sources.list.d/oneAPI.list

# 3. Install the OpenCL CPU runtime and its depenencies
apt update && apt install -y intel-oneapi-runtime-opencl intel-oneapi-compiler-dpcpp-cpp-runtime

# 4. Get the open-source compiler from GitHub
wget https://github.com/intel/llvm/releases/download/nightly-2024-06-03/sycl_linux.tar.gz
tar -xzf sycl_linux.tar.gz

# 5. Set the oneAPI environment to pick up the OpenCL CPU runtime
source /opt/intel/oneapi/setvars.sh

# 6. Run sycl-ls from the open-source compiler
LD_LIBRARY_PATH=$PWD/lib:$LD_LIBRARY_PATH ./bin/sycl-ls

The last command prints this on my system:

[opencl:cpu][opencl:0] Intel(R) OpenCL, 12th Gen Intel(R) Core(TM) i9-12900K OpenCL 3.0 (Build 0) [2024.17.5.0.08_160000.xmain-hotfix]

If you have the oneAPI base toolkit already installed, it includes both packages I installed above, so you should be able to just do steps 4-6. If that doesn’t work, perhaps there’s something else installed in your system that interferes with the above. Let me know if that worked for you.

1 Like
5

Thanks a lot @rbielski ! I now installed intel-oneapi-compiler-dpcpp-cpp-runtime, tried sycl-ls → nothing. Specifying LD_PRELOAD as in previous comment, still nothing. But then, I sourced oneAPI and now (in the new nightly) I get as expected:

$ LD_LIBRARY_PATH=$PWD/lib:$LD_LIBRARY_PATH ./bin/sycl-ls
[opencl:fpga][opencl:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device OpenCL 1.2  [2024.17.5.0.08_160000.xmain-hotfix]
[opencl:cpu][opencl:1] Intel(R) OpenCL, 11th Gen Intel(R) Core(TM) i5-11400 @ 2.60GHz OpenCL 3.0 (Build 0) [2024.17.5.0.08_160000.xmain-hotfix]
[opencl:gpu][opencl:2] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics 730 [0x4c8b] OpenCL 3.0 NEO  [22.28.23726.1]

Great!

1 Like