Rstan, parallel, log_prob from inside function

Trying to parallelise an optimization routine using rstan, splitting the data up over separate workers. It works well, but I can’t get it to work as part of a function – any pointers? The following code breaks when it is wrapped in the test function, but works fine if the function definition part is removed.
edit: @bgoodri hope you don’t mind being tagged…

test <- function(){
  real myval;
  sm <- stan_model(model_code=st)
  smf<-c() #placeholder so we can assign to it from a function
  clusterApply(cl,1:2,function(x) {
    standat$myval <<- x #CHANGE STANDATA FOR EACH PROCESS
    smf <<- sampling(object=sm,data=standat,iter=0,chains=0,init=0,check_data=FALSE, #regenerate stanfit object
  parLapply(cl,1:2,function(x) log_prob(smf,upars=1.2[c()],adjust_transform=TRUE, gradient=TRUE)) #get log prob for each process


My understanding of advanced R is limited, but this seems very fishy and may break due to some environment magic. When I use clusterApply I let the the function return the value I care about, and clusterApply returns the list of the individual results e.g.:

smf <- clusterApply(cl,1:2,function(x) {
    standat$myval <<- x #CHANGE STANDATA FOR EACH PROCESS
    sampling(object=sm,data=standat,iter=0,chains=0,init=0,check_data=FALSE, #regenerate stanfit object

Hope that helps.

that was the magic bit that lets me parallelise though, otherwise there is a lot more serialization, repeated splitting of data, repeated model generation. I don’t think that is the issue I think it’s some rcpp pointer issue that I don’t understand, but I may be wrong :)

I think it is more that the rstan package only looks for stanmodel objects in particular places, and that may not be suitable for some setups. I think it would work if you compiled the model once where the auto_write = TRUE argument is specified and then let the child processes find that on the disk when you do stan_model("file.stan").