Cannot find -lOpenCL

A model object in cmdstanpy can build just fine, until I try to bring in my GPU. The compiler error says it can’t detect c++17, but I do have it.

Here’s the code that triggers the error:

model_code = os.path.join('.', 'hw8.stan')
model = CmdStanModel(stan_file=model_code,   cpp_options = 
                     { 'STAN_THREADS' : True ,
                   'STAN_OPENCL': True,
                   'OPENCL_PLATFORM_ID':0,
                   'OPENCL_DEVICE_ID':0
                  })

Here’s the big error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[45], line 2
      1 model_code = os.path.join('.', 'hw8.stan')
----> 2 model = CmdStanModel(stan_file=model_code,  
      3                      cpp_options = 
      4                      {
      5                          'STAN_THREADS' : True ,
      6                    'STAN_OPENCL': True,
      7                    'OPENCL_PLATFORM_ID':0,
      8                    'OPENCL_DEVICE_ID':0
      9                   })

File ~/miniconda3/envs/stan/lib/python3.12/site-packages/cmdstanpy/model.py:257, in CmdStanModel.__init__(self, model_name, stan_file, exe_file, force_compile, stanc_options, cpp_options, user_header, compile)
    254         get_logger().debug("TBB already found in load path")
    256 if compile and self._exe_file is None:
--> 257     self.compile(force=str(compile).lower() == 'force', _internal=True)

File ~/miniconda3/envs/stan/lib/python3.12/site-packages/cmdstanpy/model.py:465, in CmdStanModel.compile(self, force, stanc_options, cpp_options, user_header, override_options, _internal)
    462         else:
    463             self._compiler_options.add(compiler_options)
--> 465 self._exe_file = compilation.compile_stan_file(
    466     str(self.stan_file),
    467     force=force,
    468     stanc_options=self._compiler_options.stanc_options,
    469     cpp_options=self._compiler_options.cpp_options,
    470     user_header=self._compiler_options.user_header,
    471 )

File ~/miniconda3/envs/stan/lib/python3.12/site-packages/cmdstanpy/compilation.py:481, in compile_stan_file(src, force, stanc_options, cpp_options, user_header)
    474     if 'PCH' in console or 'precompiled header' in console:
    475         get_logger().warning(
    476             "CmdStan's precompiled header (PCH) files "
    477             "may need to be rebuilt."
    478             "Please run cmdstanpy.rebuild_cmdstan().\n"
    479             "If the issue persists please open a bug report"
    480         )
--> 481     raise ValueError(
    482         f"Failed to compile Stan model '{src}'. " f"Console:\n{console}"
    483     )
    484 return str(exe_target)

ValueError: Failed to compile Stan model '/home/taylor/ds-6040/assignments/hw8/solutions/hw8.stan'. Console:
stan/lib/stan_math/make/compiler_flags:145: "Stan cannot detect if your compiler has the C++17 standard. If it does, please set STAN_HAS_CXX17=true in your make/local file. C++17 support is mandatory in the next release of Stan. Defaulting to C++14"

--- Translating Stan model to C++ code ---
bin/stanc --filename-in-msg=hw8.stan --o=/home/taylor/ds-6040/assignments/hw8/solutions/hw8.hpp /home/taylor/ds-6040/assignments/hw8/solutions/hw8.stan

--- Compiling C++ code ---
/home/taylor/miniconda3/envs/stan/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/taylor/miniconda3/envs/stan/include -std=c++1y -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes   -I stan/lib/stan_math/lib/opencl_3.0.0  -DSTAN_THREADS -I /home/taylor/miniconda3/envs/stan/include/    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/taylor/miniconda3/envs/stan/include    -DBOOST_DISABLE_ASSERTS  -DSTAN_OPENCL -DOPENCL_DEVICE_ID=0 -DOPENCL_PLATFORM_ID=0 -DCL_HPP_TARGET_OPENCL_VERSION=120 -DCL_HPP_MINIMUM_OPENCL_VERSION=120 -DCL_HPP_ENABLE_EXCEPTIONS -Wno-ignored-attributes -DINTEGRATED_OPENCL=0 -DTBB_INTERFACE_NEW  -DTBB_INTERFACE_NEW     -c  -x c++ -o /home/taylor/ds-6040/assignments/hw8/solutions/hw8.o /home/taylor/ds-6040/assignments/hw8/solutions/hw8.hpp

--- Linking model ---
/home/taylor/miniconda3/envs/stan/bin/x86_64-conda-linux-gnu-c++ -fvisibility-inlines-hidden -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/taylor/miniconda3/envs/stan/include -std=c++1y -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes   -I stan/lib/stan_math/lib/opencl_3.0.0  -DSTAN_THREADS -I /home/taylor/miniconda3/envs/stan/include/    -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.84.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem /home/taylor/miniconda3/envs/stan/include    -DBOOST_DISABLE_ASSERTS  -DSTAN_OPENCL -DOPENCL_DEVICE_ID=0 -DOPENCL_PLATFORM_ID=0 -DCL_HPP_TARGET_OPENCL_VERSION=120 -DCL_HPP_MINIMUM_OPENCL_VERSION=120 -DCL_HPP_ENABLE_EXCEPTIONS -Wno-ignored-attributes -DINTEGRATED_OPENCL=0 -DTBB_INTERFACE_NEW  -DTBB_INTERFACE_NEW    -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/taylor/miniconda3/envs/stan/lib -Wl,-rpath-link,/home/taylor/miniconda3/envs/stan/lib -L/home/taylor/miniconda3/envs/stan/lib       -Wl,-L,"/home/taylor/miniconda3/envs/stan/lib/" -Wl,--disable-new-dtags -Wl,-rpath,"/home/taylor/miniconda3/envs/stan/lib/"      /home/taylor/ds-6040/assignments/hw8/solutions/hw8.o src/cmdstan/main_threads_opencl.o       -ltbb -lOpenCL  stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a   -o /home/taylor/ds-6040/assignments/hw8/solutions/hw8
/home/taylor/miniconda3/envs/stan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lOpenCL: No such file or directory
collect2: error: ld returned 1 exit status
make: *** [make/program:80: /home/taylor/ds-6040/assignments/hw8/solutions/hw8] Error 1
rm /home/taylor/ds-6040/assignments/hw8/solutions/hw8.o /home/taylor/ds-6040/assignments/hw8/solutions/hw8.hpp

Command ['make', 'STANCFLAGS+=--filename-in-msg=hw8.stan', 'STAN_THREADS=True', 'STAN_OPENCL=TRUE', 'OPENCL_PLATFORM_ID=0', 'OPENCL_DEVICE_ID=0', '/home/taylor/ds-6040/assignments/hw8/solutions/hw8']
	error during processing No such file or directory

Operating System: Ubuntu 22.04.4
Interface Version: cmdstanpy.version is ‘1.2.4’
cmdstan version is (2, 35)
Compiler/Toolkit: g++ (Ubuntu 13.1.0-8ubuntu1~22.04) 13.1.0

Perhaps it’s due to my old GPU?

(stan) taylor@mean-machine:~$ nvidia-smi
Thu Jun 27 18:41:05 2024       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.256.02   Driver Version: 470.256.02   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:03:00.0 Off |                    0 |
| N/A   29C    P8    26W / 149W |      5MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K80           Off  | 00000000:04:00.0 Off |                    0 |
| N/A   27C    P8    29W / 149W |      5MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2044      G   /usr/lib/xorg/Xorg                  3MiB |
|    1   N/A  N/A      2044      G   /usr/lib/xorg/Xorg                  3MiB |
+-----------------------------------------------------------------------------+

I think I’m using nvidia-driver-470 here.

The C++17 standard is just a warning, the error in your logs above is:

/home/taylor/miniconda3/envs/stan/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find -lOpenCL: No such file or directory
collect2: error: ld returned 1 exit status

Which indicates that the OpenCL library wasn’t found

Woops, I missed that.

Anyway I thought opencl comes with the nvidia drivers. Running clinfo shows a bunch of stuff.

How can I add -l flags from Python? Also, is this complicated by my running everything on Anaconda? Something like this solution (but this is R not Python): cmdstanr/OpenCL error - #2 by rok_cesnovar

@WardBrian - any experience with the opencl setup for cmdstanpy?

Not personally, but the basic idea Rok laid out in R should apply if you pass an extra item in the cpp_options dictionary