ComputeCpp supports pocl on ARM CPU board?

k-ojima · April 10, 2020, 7:27am

Hi
Thanks.
I have completed cmake but make failed.

[ 1%] Building ComputeCpp integration header file /home/k-ojima/build/samples/vptr_vptr.cpp.sycl
In file included from /home/k-ojima/computecpp-sdk/samples/vptr.cpp:29:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/CL/sycl.hpp:1:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/CL/…/SYCL/sycl.hpp:20:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/SYCL/sycl_builtins.h:27:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/SYCL/cpp_to_cl_cast.h:12:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/SYCL/deduce.h:25:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/SYCL/cl_types.h:23:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/SYCL/common.h:19:
In file included from /opt/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/include/SYCL/predefines.h:541:
/usr/lib/gcc-cross/aarch64-linux-gnu/5.4.0/…/…/…/…/include/c++/5.4.0/cmath:41:10: fatal error:
‘bits/c++config.h’ file not found
#include <bits/c++config.h>
^~~~~~~~~~~~~~~~~~
1 error generated.
samples/CMakeFiles/vptr_vptr.cpp_0_ih.dir/build.make:62: recipe for target ‘samples/vptr_vptr.cpp.bc’ failed

c++config.h is located at /usr/aarch64-linux-gnu/include/c++/5/aarch64-linux-gnu/bits in my environment so I think include path have something wrong. I am looking for the source that set the path but I have not found it yet. Or there are another way to solve it?

duncan · April 10, 2020, 12:17pm

Hi,
I have seen problems when cross-compiling before that result in this sort of error. The first to check is the OpenCL headers being found in a “system” location, like /usr/include (most distributions I’ve seen will install them to /usr/include/CL). This can cause header search path problems, where the order they are used changes. To fix this, I normally use a separate directory with a known copy of the OpenCL headers, and specify them with a CMake cache variable like -DOpenCL_INCLUDE_DIR.

I hope this helps,
Duncan.

k-ojima · April 14, 2020, 8:42am

Hi

Thanks. I have solved the problem of include path but I got new error that seems to relate setting of ComputeCpp.

I specified libOpenCL.so path to computecpp but it seems to also need libOpenCL.so.1 and so on. But pocl library is only libOpenCL.so. Can I change computecpp setting to link libOpenCL.so only?

[ 1%] Built target vptr_vptr.cpp_0_ih
[ 2%] Linking CXX executable vptr
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libOpenCL.so.1, needed by /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libOpenCL.so.1, needed by /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libpthread.so.0, needed by /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so, not found (try using -rpath or -rpath-link)

tluisrs · April 14, 2020, 10:24am

Hi,

Unfortunately you can’t do this out-of-the box. It is not a problem with ComputeCpp per se. Let me explain. When we build ComputeCpp, we use the upstream Khronos OpenCL ICD loader, in other words, we use the official libOpenCL.so as provided here. When you build that you get the following files

libOpenCL.so -> libOpenCL.so.1
libOpenCL.so.1 -> libOpenCL.so.1.2
libOpenCL.so.1.2

When you link your application/library against libOpenCL.so, the linker will actually use the library SONAME, which in this case will be libOpenCL.so.1.

From what you’re saying it seems that pocl is not ICD compatible, since they provide their whole implementation in libOpenCL.so.

With that said, I have two solutions that can work for you:

The easiest solution for you would be to cp libOpenCL.so libOpenCL.so.1 or create a symlink.
Another, more complicated, but better solution would be to create a file in /etc/OpenCL/vendors called pocl.icd containing the full path to your libOpenCL.so. By doing this, you will have installed pocl in an ICD compatible way.

k-ojima · April 15, 2020, 12:02am

Thank you for your reply.

Sorry, I misunderstood the library is only one because I don’t know the ICD spec well. After knowing that, I confirmed libraries of pocl again, there are 3 files like you said.

So I put the libraries and I tried to make again, but an another error outs.

[ 1%] Built target vptr_vptr.cpp_0_ih
[ 2%] Linking CXX executable vptr
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld:/opt/pocl/libOpenCL.so: file format not recognized; treating as linker script
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld:/opt/pocl/libOpenCL.so:1: syntax error
collect2: error: ld returned 1 exit status

May be it is not problem of ComputeCpp, can you tell me comment or advice if you have any idea.

tluisrs · April 15, 2020, 8:24am

It looks like you are trying to link the wrong version of pocl, by wrong I mean one that was not cross-compiled for ARM in your case. Can you double check your pocl library is the ARM version?

You can check this by running the command

file /opt/pocl/libOpenCL.so

You should see something like the following

ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked

If you see x86_64 it means you are trying to link against the wrong version.

k-ojima · April 16, 2020, 1:54am

Thanks!

I had confirmed that the lib are correct with the file command. I just wonder why .so and .so2 say simply “data”.

/opt/pocl/libOpenCL.so: data
/opt/pocl/libOpenCL.so.2: data
/opt/pocl/libOpenCL.so.2.5.0: ELF 64-bit LSB shared object, ARM aarch64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=74136f9675d4262bda2c03fd6c3883fdd733ec53, not stripped

k-ojima · April 20, 2020, 8:07am

I have solved problems of libOpenCL.so link but another link error appears.

/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libdl.so.2, needed by /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libpthread.so.0, needed by /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libz.so.1, needed by /home/k-ojima/pocllib64/lib64/libOpenCL.so.1, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: librt.so.1, needed by /home/k-ojima/pocllib64/lib64/libOpenCL.so.1, not found (try using -rpath or -rpath-link)
/usr/lib/gcc-cross/aarch64-linux-gnu/5/…/…/…/…/aarch64-linux-gnu/bin/ld: warning: libtinfo.so.5, needed by /home/k-ojima/pocllib64/lib64/libOpenCL.so.1, not found (try using -rpath or -rpath-link

I want to link the libs by-rpath but I could not find cmake file where set -rpath in the computecpp sdk.

rod · April 20, 2020, 9:14am

Can you update your post with the latest build command you are using please?

k-ojima · April 21, 2020, 2:37am

I use below.

cmake -DComputeCpp_DIR=…/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64 -DComputeCpp_HOST_DIR=…/ComputeCpp-CE-1.3.0-Ubuntu-16.04-x86_64 -DOpenCL_LIBRARY=/home/k-ojima/pocllib64/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/include -DCMAKE_TOOLCHAIN_FILE=./cmake/toolchains/gcc-generic.cmake -DCOMPUTECPP_BITCODE=spir64 …

rod · April 21, 2020, 8:08am

It looks like you are pointing to the wrong library file. Note that the actual pocl library you need, the ARM one, is located in /opt/pocl/libOpenCL.so, the one you are linking is located in /home/k-ojima/pocllib64/lib64/libOpenCL.so, I think that these files are different.

k-ojima · April 23, 2020, 2:18am

Thanks.

I have added linker flags to CMakelists.txt then solved the problem that linker cannot found libdl.so.2 and so on.

computecpp-sdk/samples/CMakelists.txt

SET(CMAKE_EXE_LINKER_FLAGS
“${CMAKE_EXE_LINKER_FLAGS} -Wl,-rpath,/usr/aarch64-linux-gnu/lib”)

But new errors appears. libComputeCpp.so seems to be not able to find OpenCL API symbols although pocl has their symbols, (I confirmed libOpenCL.so has them with"nm" commmand ).

/home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so: undefined reference to clReleaseSampler@OPENCL_1.0' /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so: undefined reference to clEnqueueWriteImage@OPENCL_1.0’
/home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so: undefined reference to clReleaseMemObject@OPENCL_1.0' /home/k-ojima/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64/lib/libComputeCpp.so: undefined reference to clRetainEvent@OPENCL_1.0’
(snip)

tluisrs · April 23, 2020, 8:50am

There should be no need to change the rpath of the executable, CMake should take care of this for you. Could you try to just the following?

cmake -DComputeCpp_DIR=…/ComputeCpp-CE-1.3.0-Ubuntu-16.04-ARM_64 -DComputeCpp_HOST_DIR=…/ComputeCpp-CE-1.3.0-Ubuntu-16.04-x86_64 -DOpenCL_LIBRARY=/opt/pocl/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/include -DCMAKE_TOOLCHAIN_FILE=./cmake/toolchains/gcc-generic.cmake -DCOMPUTECPP_BITCODE=spir64

This is where your pocl library is located. If this doesn’t work could you paste the output of the following command here?

nm /opt/pocl/libOpenCL.so | grep OPENCL_1.0

k-ojima · April 30, 2020, 2:28am

Thanks.

Unfortunately, in my environment, I followed cmake command but cmake failed, so in addition I edited cmake files to specify cross-compiler options.

And I do the nm command you said, then there are no output. I understood ComputeCpp expects OpenCL APIs with the OPENCL_1.0 but pocl lib does not have it. Can I make the ComputeCpp ignore the OPENCL_1.0?

duncan · April 30, 2020, 4:49pm

Which CMake command failed, exactly?

I’ve not used pocl on ARM myself but I have both compiled ComputeCpp applications for ARM and used pocl a few times and not run into trouble. If I were attempting to do the same thing you were doing, I would take the following steps. While they are something of a restatement of what has been discussed before, I don’t expect any individual step to fail, and following them might help us pinpoint the error:

Download OpenCL Headers from GitHub
Download OpenCL ICD Loader, again from GitHub
Build this for ARM platform using same headers
Build and link my ComputeCpp application using the same toolchain and the OpenCL ICD loader from the previous step (here you would specify -DOpenCL_INCLUDE_DIR and -DOpenCL_LIBRARY as pointing to the downloaded headers and just-built OpenCL ICD Loader library
Now you can deploy to the device, pocl has ICD loader support as described on this page so you should be able to add the entry to /etc/OpenCL/vendors and from there it should just work. Alternatively you could preload the pocl library, that would work too.

I did notice in one of your replies that OpenCL_INCLUDE_DIR was set as /usr/include - I would really strongly recommend against that. And to be clear, I would not use pocl in the build steps for the SYCL code, and would rather stick to the plain ICD loader library.

If you tell us which of these stages has an error and what that is, maybe we can pinpoint what’s going wrong - as I said before, I don’t expect any of these stages to fail.

All that being said, if you do something like this then it should work. I did it just the other day to get some profiling data from an ARM device.

k-ojima · May 13, 2020, 1:54am

Sorry for late reply. Thank you.

I followed your procedures and now I am stuck in below.

Now you can deploy to the device, pocl has ICD loader support as described on this page so you should be able to add the entry to /etc/OpenCL/vendors and from there it should just work. Alternatively you could preload the pocl library, that would work too.

I try to execute the vptr sample on a target board, then errors appear.

root@s32v234sbc:/home/samples# ./vptr
./vptr: /home/sdk/drivers/libOpenCL.so.1: no version information available (required by /home/sdk/drivers/libComputeCpp.so)
./vptr: /home/sdk/drivers/libOpenCL.so.1: no version information available (required by /home/sdk/drivers/libComputeCpp.so)
./vptr: /home/sdk/drivers/libOpenCL.so.1: no version information available (required by /home/sdk/drivers/libComputeCpp.so)
attempting to add vendor /usr/local/lib/libpocl.so.2.5.0…
successfully added vendor /usr/local/lib/libpocl.so.2.5.0 with suffix POCL
terminate called after throwing an instance of ‘cl::sycl::compile_program_error’
Aborted
root@s32v234sbc:/home/samples#

Do you have any idea for this?

rod · May 14, 2020, 8:08am

Are you able to run clinfo on the device? If yes it would be useful to see what the OpenCL drivers are reporting in terms of what is supported.
Can you show us the output of clinfo?

duncan · May 14, 2020, 5:27pm

Specifically here the error means that the driver (which appears to be pocl, which is promising) wasn’t able to compile the code. The parallel-for sample has better error handling than the vptr sample; if you run it you should get full build log. Of course if it runs then it means something in the vptr sample specifically is causing the problem.

Actually a better thought is that the configuration file will allow you to select verbose output - if I remember correctly, this will cause the build log to be output to standard output directly, regardless of the user code. It’s too noisy to be enabled by default but might help here.

k-ojima · May 20, 2020, 8:38am

Thank you. Finally I’ve done combination of ComputeCpp and pocl on arm CPU board!

Cause of the latest error is that I did not set a pocl compile option.

if(ENABLE_SPIR)
set(HOST_DEVICE_EXTENSIONS “${HOST_DEVICE_EXTENSIONS} cl_khr_spir”)
endif()

I set ENABLE_SPIR, then computecpp sample works properly.

Thank you for your kind cooperation.

duncan · May 21, 2020, 3:12pm

Great news! I’m glad it’s working. Let us know if you run into any other problems