I wanted to implement that, but haven’t found the time yet…but if you got trouble simulating your data then you probably anyway suffer a lot during fitting and in that case it’s fine to invest some manual work in parallelizing the gq phase yourself…it’s tricky with rng seeds and stuff to get it right and there are bigger things to wrangle in Stan at the moment, I think.
…but parallel gq is a nice project …
Oh right, it’s not that trivial if you have to take care of parallel rng, didn’t think about that…
@Philipp-Neubauer you might be able to exploit the efficiency of stans compilation and use all of your cores by “manually” splitting up the output csvs into as many as you have cores (and hack them a bit) , and then run gq on those files.
Thanks @wds15 and @Funko_Unko for the suggestions to split the GQ block out and use the standalone method - that works a charm (after I moved all transformed pars into the parameter block). It’s reasonably quick without running it in parallel (which makes me wonder why it has such a bad effect on threading when I run the model with GQ in MCMC mode).
Hi guys,
I’ve been following this thread with much interest. @Philipp-Neubauer would you be so kind as to post your final, working, solution. Also, for those (few) of us that are not R users, could you perhaps show how to run your model with cmdstan?
Very best wishes,
Sergi
It should actually be possible to run the gq during sampling while sampling proceeds. It’s just that this would require a few changes like independent rngs for the sampler and the gq part and in addition one would have to take care of the output in a clever way.
It’s so much easier to do gq parallelization manually right now.
For cmdstan there is a gq facility as well.