OpenCL 2.0

[stevebronder]

@Bob_Carpenter

I’d be reluctant to adopt a GPU standard that Nvidia isn’t supporting yet.

This weekend I’d like to reach out to the email at the end of that presentation and ask what their status is for 2.0 to move out of beta.

@Bob_Carpenter

The free memory sharing does sound too good to be true. I can believe it’s coded that way, but I don’t see how it could perform that way.

I’m slowly reading through the OpenCL spec and code to see how they do this or specify how to do this. Hand-waving a guess, if you have one piece of virtual memory pointing to two different pieces of physical memory I think what they are saying is not impossible.

@Bob_Carpenter

That sounds like a big maintenance burden. We’ve tried not to have to support multiple versions of the same interface. Given how long it’s taken to get the basic GPU stuff ready to go, I’d be reluctant to try to rewrite it all at this point.

We are 100% on the same page, moving to 2.0 is not on our current plate. We are talking about investigating this for the future.

@Bob_Carpenter

[Sean] We won’t be able to make much (any?) use of this without rewriting our Math library to use MatrixBase instead of Matrices everywhere.

I think that’s the plan going forward in any case.

Here is a link to Dan in the thread about input checks talking about how stan had issues with MatrixBase in the past. In the linked google discussion Ben made it sound like we could get around the coefficient access issue Dan mentioned though.

@seantalts

Are there many of those? My impression was most powerful GPUs are not integrated.

Intel has this “visual compute accelerator” which is an integrated CPU+GPU with 3 Xeon processors and a P580 GPU. The gpu is not crazy powerful like a volta, but since they are integrated you don’t pay a transfer cost which is rad.

The gpus are not that powerful, but the last time we did a big profiling of the gpu code I think Rok found that 70% of the time was spent doing memory transfers. If the CPU+GPU combo can dramatically reduce those transfer costs then I think we would end up with a pretty decent speedup. Back of the hand math, if the gpu process was 100ms and we reduce memory transfer costs by 95% while increasing the algorithm cost by 15% then I think we would end up with a 2.6x speedup relative to the current version. The current gpu code is about 8x faster so the gpu version with svm for 5K matrices would be 21x faster than the cpu version.

All those numbers are made up though so we would need to get our hands on one of these things to see what sort of speedups we would get.