I was able to verify that that the serialization error above comes is due to being out of memory. I used the free
command to log memory usage and sure enough I ran out,
nohup watch -n 1 'free -mh' > mem.log &
I noticed in another thread that either Bob or Ben said rstan
allocates memory up front while CmdStan
does not (it streams?). I think I’ll give that a shot. I also might give the parallel::mcapply
and sampling()
route a shot: https://github.com/stan-dev/rstan/issues/243#issuecomment-164560369