Stan crashes all the time


#1

I am running a simulation study with rstan. It ran really well until recently my rstan get crashed on my PC even with the same simulation condition, which ran just fine one week or two ago. The error message goes like:

Error in unserialize(socklist[[n]]) : error reading from connection
Error in serialize(data, node$con, xdr = FALSE) :
error writing to connection

I read a lot on the forum about similar issues, but all the solutions that may work for others does not work for me at all. Now I cannot run any condition on my work PC, which drives me crazy.

Please help!


#2

Are you calling stan or sampling from inside a loop or wrapper function or something. If so, you need to be very careful to not reload the shared object multiple times. That usually entails compiling the Stan program once on the outside and then calling sampling on it repeatedly.


#3

Yes, I did place the Stan program inside the loop function, which is placed inside another function. This is because I want to update the data for the Stan program for each replication. There is no shared objects for each replication, or more accurately, the shared objects are hard coded and placed outside of the loop.

I just wonder if it is a Stan compiling issue or what.

I have conditions of 5 items, 10 items, and 20 items, calibrated with multilevel two-parameter logistic model and one-parameter logistic model, respectively. For both models, the 5- and 10- item conditions compiled just fine, while for the 20-item condition, it crashes 99% percent of the time with about 1% of time it does not crash.


#4

That will cause errors (not crashes) 99% of the time. It is not necessary to put the Stan program inside a function inside a loop in order to update the data for each replication. You can compile the model once on the outside with something like

library(rstan)
sm <- stan_model("program.stan")

and then do your loop calling sampling(sm, data = ...).


#5

Cool. Let me try. Thank you so much, Dr Goodrich. But it does crash my Stan program because it stops rstan from running with the message I had in the initial post. It does not have problems with less number of items though.


#6

Are you by chance running

Sys.setenv(LOCAL_CPPFLAGS = '-march=native')

like the start up messages suggest?

I ran into this serialization error. I fixed it by not running that command.


#7

I do not. Actually I had before, but deleted it. Do I have to reset anything if I had run it before? Thank you very much.


#8

No, it resets every time you start R. But I think your problem with is with the looping if it works once regardless of what LOCAL_CPPFLAGS is.


#9

OK. I will give it a try and let you know.

I also got the following error message occasionally:

Warning message:
In new_CppObject_xp(fields$.module, fields$.pointer, …) :
NAs introduced by coercion to integer range

What does that mean?


#10

Integers overflowed


#11

OK. What are consequences? Will it affect my estimation results?


#12

It means numbers that are bigger than 2^31 - 1 become negative integers, so that would ruin everything.


#13

I also found that my Stan program is read in from external file, only the model fitting code is in the loop function as shown below:

fit1 <- stan(
  file = paste0(path,model),  # Stan program
  data = data,    # named list of data
  seed=7030507896,
  chains = 4,            # number of Markov chains
  warmup = 1000,          # number of warmup iterations per chain
  iter = 2000,           # total number of iterations per chain
  #thin = 10,
  cores = 4,             # number of cores (using 2 just for the vignette)
  refresh = 1,           # show progress every 'refresh' iterations
  control = list(max_treedepth = 10,adapt_delta=0.85)
)

model is read in externally, only data is changing for each loop.

So I think it may not be the looping issue.


#14

Oh men~~~Not a good sign. Thank you Dr Goodrich. :-)


#15

That will cause the DLL to get reloaded every time, which will eventually cause problems.


#16

Good to know. I will place the model in the code, outside of the loop. Thank you very much.