Variational inference with OpenCL/GPUs

I’m trying to run a model where I define the log likelihood of a model using a user-defined function that involves a lot of matrix math (multiplications and inverses). A GPU is installed on my machine and OpenCL recognises it but running the variational algorithm offers no speedup at all, when run as:

model = cmdstan_model('model.stan', cpp_options=list(stan_opencl=TRUE))
results = model$variational(
    opencl_ids=c(0, 0),

Does the variational inference not use GPUs / is there a specific way to write matrix ops such that the GPU is used for them?

I’ve posted the full code here: [FR] Adding Gaussian Process Latent Variable Model Examples · Issue #442 · stan-dev/docs · GitHub

1 Like

In fact - this tutorial Running Stan on the GPU with OpenCL • cmdstanr, doesn’t seem to work with variational.

Following up on this if anyone knows :)

Variational inference does in general work with OpenCL, there are however a few things to note here:

  • the opencl_ids argument is not currently exposed for variational in cmdstanr, which means that you need to set the IDs via the makefile OPENCL_PLATFORM_ID and OPENCL_DEVICE_ID flags at compile time. We will probably add this for variatonal as well, but it was not a big priority so far.

  • your model is very unlikely to yield a speedup from using the Stan OpenCL backend right now