Variational inference with OpenCL/GPUs

General
1

I’m trying to run a model where I define the log likelihood of a model using a user-defined function that involves a lot of matrix math (multiplications and inverses). A GPU is installed on my machine and OpenCL recognises it but running the variational algorithm offers no speedup at all, when run as:

model = cmdstan_model('model.stan', cpp_options=list(stan_opencl=TRUE))
results = model$variational(
    data=data,
    seed=42,
    iter=2000,
    opencl_ids=c(0, 0),
    algorithm='meanfield',
    eta=1,
    adapt_engaged=F
)

Does the variational inference not use GPUs / is there a specific way to write matrix ops such that the GPU is used for them?

I’ve posted the full code here: [FR] Adding Gaussian Process Latent Variable Model Examples · Issue #442 · stan-dev/docs · GitHub

1 Like
2

In fact - this tutorial Running Stan on the GPU with OpenCL • cmdstanr, doesn’t seem to work with variational.

3

Following up on this if anyone knows :)

4

Variational inference does in general work with OpenCL, there are however a few things to note here:

  • the opencl_ids argument is not currently exposed for variational in cmdstanr, which means that you need to set the IDs via the makefile OPENCL_PLATFORM_ID and OPENCL_DEVICE_ID flags at compile time. We will probably add this for variatonal as well, but it was not a big priority so far.

  • your model is very unlikely to yield a speedup from using the Stan OpenCL backend right now