Sampler error in rstan on Mac that doesn't reproduce on other computers

I’m running a hierarchical model in rstan (v. 2.15.1, R version 3.4.0), and frequently – but not consistently – I get an error right after compilation:

Iteration: 1 / 500 [ 0%] (Warmup)
[1] “Error in sampler$call_sampler(args_list[[i]]) : "
[2] " c++ exception (unknown reason)”
error occurred during calling the sampler; sampling not done

I’ve tried to strip it down to the minimum that produces this error:

gmodel='data {
int<lower=0> n; // number of subjects
int<lower=0> k; // number of observations per subject
real r;
real y[k,n];
}

parameters {
real<lower=0> tau[n];
real<lower=0> beta;
}

transformed parameters {
real<lower=0> sigma[n];
real<lower=0> betas;
betas=beta+r;
for (i in 1:n){
sigma[i]=pow(tau[i],-.5);
}
}
//

model {
beta ~ exponential(1);
for(i in 1:n){
tau[i] ~ exponential(betas);
for (j in 1:k){
y[j,i] ~ normal(0,sigma[i]);
}
}
}’

nn=200
tau=rexp(nn,2)
kk=3
yy=sapply(tau,function(t) rnorm(kk,0,t^(-.5)))

gamdat=list(n=nn,y=yy,k=kk,r=.1)
gamstan=stan(model_code = gmodel,data = gamdat,iter=500,chains=1)

The transformed parameter betas is there because I thought maybe there was a problem with sampling negative values of beta. Indeed, setting r=1 – so betas=beta+1 – makes it run consistently, while r = .1 or anything lower pretty consistently crashes. But the code might run several times in a row, then crash on the fourth try, though it’s the same data each time. Once it crashes, it crashes immediately each time I run it again, until I force it to recompile by changing the model.

I was originally running it on RStan 2.16.2, but I reinstalled the older version because I thought that might be causing the problem, because it was compiled on R 3.4.1. But no change.

To make it even more confusing, I tried running exactly the same code on a Linux server, also with R 3.4.0 and RStan 2.16.2, and there it runs without a hitch. Obviously I could run everything on the server, and I guess I will, but I’d appreciate hearing if anyone has any insight into this.

It usually has to do with a different version or configuration of Rcpp than the one being used to compile the Stan model.

Thanks. How could I tell which version of Rcpp is the right one? I’m now running the latest version (2.16.2) of rstan and the latest version (0.12.11) of Rcpp (dated 22 May 2017). I also updated to R version 3.4.1, with no change.

I would just install the latest Rcpp from source and then try your Stan stuff again.

Okay. I reinstalled R, Rcpp, and rstan, with no change.

What is in your ~/.R/Makevars file?

CC = /usr/local/clang4/bin/clang
CXX = /usr/local/clang4/bin/clang++
LDFLAGS=-L/usr/local/clang4/lib
FLIBS=-L/usr/local/gfortran/lib

I put the first three in a few months ago, under advice, after some incompatibility broke Rcpp. It fixed the problem at the time.

The last line I added recently — so perhaps it’s related to the problem — after Rcpp started complaining about inability to find gfortran. That, together with updating to the newest version of gfortran repaired it.

Best wishes,

David

I am out of ideas. Maybe @jonah can reproduce it?

Upon further review, I have another idea. Is there a .rds file in the same directory as the .stan file that was compiled with a previous version of RStan?

Not in the same directory, though in nearby directories. Also, I’m running the stan code for simplicity directly from the console, without calling a file.

I’ll try this afternoon and report back.

When I copy the code exactly from the original post and run it using 2.15 I get

Rejecting initial value:
  Error evaluating the log probability at the initial value.

Initialization between (-2, 2) failed after 100 attempts. 
 Try specifying initial values, reducing ranges of constrained values, or reparameterizing the model.
[1] "Error in sampler$call_sampler(args_list[[i]]) : Initialization failed."
error occurred during calling the sampler; sampling not done

which is different than the error originally reported.

I’m terribly sorry, somehow the simplified code that I copied in lost the <lower=0> restrictions, which spoiled the initialisation. I’ve put them back in in the code below. This code produces the error I stated when run on the Mac, and no error when run on the Linux server.

gmodel='data {
int n; // number of subjects
int k; // number of observations per subject
real r;
real y[k,n];
}
parameters {
real<lower=0> tau[n];
real<lower=0> beta;
}

transformed parameters {
real sigma[n];
real<lower=0> betas;
betas=beta+r;
for (i in 1:n){
sigma[i]=pow(tau[i],-.5);
}
}
//

model {
beta ~ exponential(1);
for(i in 1:n){
tau[i] ~ exponential(betas);
for (j in 1:k){
y[j,i] ~ normal(0,sigma[i]);
}
}
}’

nn=200
tau=rexp(nn,2)
kk=3
yy=sapply(tau,function(t) rnorm(kk,0,t^(-.5)))

gamdat=list(n=nn,y=yy,k=kk,r=.1)
gamstan=stan(model_code = gmodel,data = gamdat,iter=500,chains=1)

No worries. Unfortunately I have no idea why that problem is happening when you’re running it on your Mac. I just ran your updated code and it runs fine for me on Mac (OS X 10.12.5) with both RStan 2.15.1 and RStan 2.16.2.

Thanks for trying. It’s pretty mysterious. I should just mention, for the record, that it doesn’t always crash when running with these parameters – this was also part of my suspecting that it might be an initialisation issue, and the reason why I put in the r parameter for testing, to see the effect of shifting the distribution. Indeed, I just checked, and there is no crash when I put in seed=1, but there is a crash when seed=2. It always crashes when r=0, never crashes when r=1.

One more thing, possibly irrelevant: When it runs successfully, it produces several warnings during compiling. I don’t know if they’re relevant:

/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/config/compiler/clang.hpp:196:11: warning: ‘BOOST_NO_CXX11_RVALUE_REFERENCES’ macro redefined [-Wmacro-redefined]
#define BOOST_NO_CXX11_RVALUE_REFERENCES
^
<command line>:6:9: note: previous definition is here
#define BOOST_NO_CXX11_RVALUE_REFERENCES 1

/Library/Frameworks/R.framework/Versions/3.4/Resources/library/BH/include/boost/multi_array/concept_checks.hpp:42:43: warning: unused typedef ‘index_range’ [-Wunused-local-typedef]
typedef typename Array::index_range index_range;

I don’t get these warnings when compiling on the server (where the runs always work), though I get a huge run of other warnings, largely of the form “warning: non-standard suffix on floating constant”.

Those Boost warnings are a red herring. I think the answer is to install rstan from source with install.packages("rstan", type = "source") on a Mac as the CRAN binary for Macs seems to have something wrong with it.

No, didn’t help…

Okay, we are going to need more people with Macs to get to the bottom of this problem. What version of Xcode do you have?

8.3.3.

I couldn’t reproduce the error (using the <lower=0>-version of the example code) on my Mac using the following software versions.

R version 3.3.2 (2016-10-31) -- "Sincere Pumpkin Patch"
rstan (Version 2.14.1, packaged: 2016-12-28 14:55:41 UTC, GitRev: 5fa1e80eb817)
Xcode 8.2.1