Sampler error in rstan on Mac that doesn't reproduce on other computers


#1

I’m running a hierarchical model in rstan (v. 2.15.1, R version 3.4.0), and frequently – but not consistently – I get an error right after compilation:

Iteration: 1 / 500 [ 0%] (Warmup)
[1] "Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)"
error occurred during calling the sampler; sampling not done

I’ve tried to strip it down to the minimum that produces this error:

gmodel='data {
int<lower=0> n; // number of subjects
int<lower=0> k; // number of observations per subject
real r;
real y[k,n];
}

parameters {
real<lower=0> tau[n];
real<lower=0> beta;
}

transformed parameters {
real<lower=0> sigma[n];
real<lower=0> betas;
betas=beta+r;
for (i in 1:n){
sigma[i]=pow(tau[i],-.5);
}
}
//

model {
beta ~ exponential(1);
for(i in 1:n){
tau[i] ~ exponential(betas);
for (j in 1:k){
y[j,i] ~ normal(0,sigma[i]);
}
}
}’

nn=200
tau=rexp(nn,2)
kk=3
yy=sapply(tau,function(t) rnorm(kk,0,t^(-.5)))

gamdat=list(n=nn,y=yy,k=kk,r=.1)
gamstan=stan(model_code = gmodel,data = gamdat,iter=500,chains=1)

The transformed parameter betas is there because I thought maybe there was a problem with sampling negative values of beta. Indeed, setting r=1 – so betas=beta+1 – makes it run consistently, while r = .1 or anything lower pretty consistently crashes. But the code might run several times in a row, then crash on the fourth try, though it’s the same data each time. Once it crashes, it crashes immediately each time I run it again, until I force it to recompile by changing the model.

I was originally running it on RStan 2.16.2, but I reinstalled the older version because I thought that might be causing the problem, because it was compiled on R 3.4.1. But no change.

To make it even more confusing, I tried running exactly the same code on a Linux server, also with R 3.4.0 and RStan 2.16.2, and there it runs without a hitch. Obviously I could run everything on the server, and I guess I will, but I’d appreciate hearing if anyone has any insight into this.


#2

It usually has to do with a different version or configuration of Rcpp than the one being used to compile the Stan model.


#3

Thanks. How could I tell which version of Rcpp is the right one? I’m now running the latest version (2.16.2) of rstan and the latest version (0.12.11) of Rcpp (dated 22 May 2017). I also updated to R version 3.4.1, with no change.


#4

I would just install the latest Rcpp from source and then try your Stan stuff again.


#5

Okay. I reinstalled R, Rcpp, and rstan, with no change.


#6

What is in your ~/.R/Makevars file?


#7

CC = /usr/local/clang4/bin/clang
CXX = /usr/local/clang4/bin/clang++
LDFLAGS=-L/usr/local/clang4/lib
FLIBS=-L/usr/local/gfortran/lib

I put the first three in a few months ago, under advice, after some incompatibility broke Rcpp. It fixed the problem at the time.

The last line I added recently — so perhaps it’s related to the problem — after Rcpp started complaining about inability to find gfortran. That, together with updating to the newest version of gfortran repaired it.

Best wishes,

David

#8

I am out of ideas. Maybe @jonah can reproduce it?


#9

Upon further review, I have another idea. Is there a .rds file in the same directory as the .stan file that was compiled with a previous version of RStan?


#10

Not in the same directory, though in nearby directories. Also, I’m running the stan code for simplicity directly from the console, without calling a file.


#11

I’ll try this afternoon and report back.


#12

When I copy the code exactly from the original post and run it using 2.15 I get

Rejecting initial value:
  Error evaluating the log probability at the initial value.

Initialization between (-2, 2) failed after 100 attempts. 
 Try specifying initial values, reducing ranges of constrained values, or reparameterizing the model.
[1] "Error in sampler$call_sampler(args_list[[i]]) : Initialization failed."
error occurred during calling the sampler; sampling not done

which is different than the error originally reported.


#13

I’m terribly sorry, somehow the simplified code that I copied in lost the <lower=0> restrictions, which spoiled the initialisation. I’ve put them back in in the code below. This code produces the error I stated when run on the Mac, and no error when run on the Linux server.

gmodel='data {
int n; // number of subjects
int k; // number of observations per subject
real r;
real y[k,n];
}
parameters {
real<lower=0> tau[n];
real<lower=0> beta;
}

transformed parameters {
real sigma[n];
real<lower=0> betas;
betas=beta+r;
for (i in 1:n){
sigma[i]=pow(tau[i],-.5);
}
}
//

model {
beta ~ exponential(1);
for(i in 1:n){
tau[i] ~ exponential(betas);
for (j in 1:k){
y[j,i] ~ normal(0,sigma[i]);
}
}
}’

nn=200
tau=rexp(nn,2)
kk=3
yy=sapply(tau,function(t) rnorm(kk,0,t^(-.5)))

gamdat=list(n=nn,y=yy,k=kk,r=.1)
gamstan=stan(model_code = gmodel,data = gamdat,iter=500,chains=1)


#14

No worries. Unfortunately I have no idea why that problem is happening when you’re running it on your Mac. I just ran your updated code and it runs fine for me on Mac (OS X 10.12.5) with both RStan 2.15.1 and RStan 2.16.2.


#15

Thanks for trying. It’s pretty mysterious. I should just mention, for the record, that it doesn’t always crash when running with these parameters – this was also part of my suspecting that it might be an initialisation issue, and the reason why I put in the r parameter for testing, to see the effect of shifting the distribution. Indeed, I just checked, and there is no crash when I put in seed=1, but there is a crash when seed=2. It always crashes when r=0, never crashes when r=1.

One more thing, possibly irrelevant: When it runs successfully, it produces several warnings during compiling. I don’t know if they’re relevant:

/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:196:11: warning: ‘BOOST_NO_CXX11_RVALUE_REFERENCES’ macro redefined [-Wmacro-redefined]
#define BOOST_NO_CXX11_RVALUE_REFERENCES
^
<command line>:6:9: note: previous definition is here
#define BOOST_NO_CXX11_RVALUE_REFERENCES 1

/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef ‘index_range’ [-Wunused-local-typedef]
typedef typename Array::index_range index_range;

I don’t get these warnings when compiling on the server (where the runs always work), though I get a huge run of other warnings, largely of the form “warning: non-standard suffix on floating constant”.


#16

Those Boost warnings are a red herring. I think the answer is to install rstan from source with install.packages("rstan", type = "source") on a Mac as the CRAN binary for Macs seems to have something wrong with it.


#17

No, didn’t help…


#18

Okay, we are going to need more people with Macs to get to the bottom of this problem. What version of Xcode do you have?


#19

8.3.3.


#20

I couldn’t reproduce the error (using the <lower=0>-version of the example code) on my Mac using the following software versions.

R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
rstan (Version 2.14.1, packaged: 2016-12-28 14:55:41 UTC, GitRev: 5fa1e80eb817)
Xcode 8.2.1