Cmdstan and opencl

Interfaces CmdStan
1

Does cmdstan support opencv? On my system the module avail gives the availability of opencv but there is no opencl module. Thus I assume that opencl libraries are not installed.

Thanks for any insight.

2

You typed “opencv” a couple of times where I think you meant “opencl”. In any event, my understanding is that Stan assumes opencl is installed if you want to use stuff like GPUs.

3

If you haven’t already I think you also need to set STAN_OPENCL=true when compiling (and possibly set the opencl device and platform ids). In CmdStanR R package we set these in make/local:

4

Thanks a lot. It seems on the system there is no opencl but there is opencv and cuda. I am not an expert in those but there is a difference between opencv, opencl, and cuda. make build complains that opencl.h is not found. I wonder if there is a way to make cmdstan so it uses opencv or cuda. It seems cuda is proprietary framework created by Nvidia and OpenCL is open source. Since the system has Nvidia GPUs they installed cuda instead of opencl.

1 Like
5

OpenCV is a computer-vision library

It has nothing to do with Stan.

6

Thanks. Then the question is how to make cmdstan so it utilizes GPUs but instead of using openCL uses cuda?

7

The OpenCL .lib or .dll files are installed with CUDA or the latest GPU drivers.

If you are running Linux try installing the clinfo (sudo apt install clinfo or equivalent should do it) tool to check if there is a valid OpenCL implementation. Same if you are on a mac os version that still supports OpenCL officially (before Mojave).

If you are on Windows you need to specificy an additional flag pointing to the OpenCL lib files. See https://github.com/stan-dev/math/wiki/OpenCL-GPU-Routines

1 Like
8

Just to add to @rok_cesnovar’s answer, all Nvidia’s card support OpenCL since quite a long time ago, so it should work, the issue is almost certainly with drivers (OpenCL drivers are sometimes not installed by default).

1 Like
9

The OpenCL .lib or .dll files are installed with CUDA or the latest GPU drivers.

Thanks for all insight. Since I can’t do sudo I need somehow to figure a way around. As is usual in universities, there are modules on Linux clusters - one of them is cuda/10.1. So hopefully OpenCL libraries exist. But after I did module load cuda the cmdstan make build complained that CL/opencl.h is not found when making stansummary. clinfo is also not around.

1 Like
10

Sounds like you should talk to your cluster admin.

1 Like
11

Yep, talking with admins take time. It appears the environment is OpenACC. Any suggestions how to make cmdstan to use GPUs in this environment?

12

I am sorry, but I don’t think there is a way to make GPUs work in Stan without OpenCL support. Also bear in mind that models that appear slow often hide some pathology and throwing more computing power at them won’t necessarily help you, while finding the right way to express your model might make the computation practical on a laptop. Judging from the other topics your are involved in, I would guess you are not having such a huge dataset neither an extremely complicated model, so GPUs are unlikely to help you much.

Best of luck with your analysis!

13

I am working on other problems as well and being able to utilize GPUs it would be nice. There is a C++ wrapper for OpenCL. I wonder if cmdstan could be linked to use C++ wrapper?

14

We use the OpenCL wrapper for C++. What version of cuda do you have installed? Can you give us more info on your operating system and other build info?

Have you followed the instructions below? If they are not clear or applicable please let us know how so we can fix them up!

http://mc-stan.org/math/opencl_support.html

15

University has cuda from 7.5 to 10.1. They also have clang. However, when I make cmdstan it tells that CL/opencl.h is not found. Since I don’t have sudo rights I wonder if I could install necessary .h and .so files in my folder. Or maybe there is another way around?

16

Are you using the current develop in cmdstan? It may be easier for you to work through cmdstanr since we have opencl stuff setup over there

Are you installing from github and cloning with git clone --recursive https://github.com/stan-dev/cmdstan?

17

Yes. I do
git clone --recursive https://github.com/stan-dev/cmdstan
Then I change make/local to

STAN_OPENCL=true
OPENCL_DEVICE_ID=0
OPENCL_PLATFORM_ID=0
and make build

18

Can you post the full error message?

19

g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -I stan/lib/stan_math/lib/opencl_2.1.0 -I stan/lib/stan_math/lib/tbb_2019_U8/include -O3 -I src -I stan/src -I lib/rapidjson_1.1.0/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.3.3 -I stan/lib/stan_math/lib/boost_1.69.0 -I stan/lib/stan_math/lib/sundials_4.1.0/include -DBOOST_DISABLE_ASSERTS -DSTAN_OPENCL -DOPENCL_DEVICE_ID=0 -DOPENCL_PLATFORM_ID=0 -DCL_HPP_TARGET_OPENCL_VERSION=120 -DCL_HPP_MINIMUM_OPENCL_VERSION=120 -DCL_HPP_ENABLE_EXCEPTIONS -Wno-ignored-attributes -c -o bin/cmdstan/stansummary.o src/cmdstan/stansummary.cpp
In file included from stan/lib/stan_math/stan/math/opencl/err/check_opencl.hpp:6:0,
from stan/lib/stan_math/stan/math/opencl/opencl_context.hpp:14,
from stan/lib/stan_math/stan/math/opencl/opencl.hpp:6,
from stan/lib/stan_math/stan/math/prim/mat/fun/mdivide_left_tri.hpp:10,
from stan/lib/stan_math/stan/math/prim/mat/fun/mdivide_left_tri_low.hpp:6,
from stan/lib/stan_math/stan/math/prim/mat/fun/chol2inv.hpp:7,
from stan/lib/stan_math/stan/math/prim/mat.hpp:71,
from stan/src/stan/mcmc/chains.hpp:5,
from src/cmdstan/stansummary.cpp:5:
stan/lib/stan_math/lib/opencl_2.1.0/cl.hpp:527:23: fatal error: CL/opencl.h: No such file or directory
compilation terminated.
make: *** [bin/cmdstan/stansummary.o] Error 1