Issue running rstan models using foreach

I am trying to run models in parallel using foreach. I have tried using Linux, Windows and Mac. So far, I only can run the models using my old mac (very slow though).

Windows 10

  • Error in unserialize(socklist[[n]]) : error reading from connection
  • I am able to run a single model using several cores (chains) without problems
  • The foreach models use chains = 1, cores = 1
  • rstan_2.21.2 , StanHeaders_2.21.0-7, rethinking_2.13, Rcpp_1.0.6, RcppParallel_5.1.4
  • No makevars file

Linux

  • *** caught segfault *** address 0x7f6e9724d120, cause 'memory not mapped'
  • I am able to run a single model using several cores (chains) without problems
  • rstan_2.21.2 , StanHeaders_2.21.0-7, rethinking_2.13, Rcpp_1.0.6, RcppParallel_5.1.4
  • Makevars:
CXX14FLAGS=-O3 -march=native -mtune=native -fPIC
CXX14=g++

The code I am using has this structure:

cl = makeCluster(clusters)
registerDoParallel(cl)
    
output = foreach(i = 1:nsamples,
        .packages = c("parallel", "doParallel", 
            "data.table", "rstan", "rethinking")) %dopar% {
    
        options(mc.cores = 1)

        model = ulam(
            flist, 
            data = mdata[[i]], chains = 1, cores = 1, 
            iter = iterations
        )

        return(model)
}

Any ideas or suggestions? Thanks!

I would guess that it has something to do with each call trying to re-use the same pre-compiled model.

Is it any different if you call : rstan_options(auto_write = FALSE) before ulam in the foreach call?

No difference :(

Thanks!

Does it also error when run sequentially? (i.e. with %do%)

With %do% seems to run fineā€¦

Using cmdstan solved the issue, thanks!