Pystan crushes with exit code 139 (interrupted by signal 11: SIGSEGV)

General
1

That’s a bit scary, but suddenly all of my models that were working previously, started to crush once the sampling is done:

Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

How can I solve it??

2

Update - I checked it on three different machines. Models weren’t changed. Different models crush in all three machines.

3

Update 2 - for models that do run, I get the following error after sampling:

/Users/venv/lib/python3.6/site-packages/pystan/diagnostics.py:213: RuntimeWarning: invalid value encountered in true_divide
  numer = ((np.diff(energies, axis=0)**2).sum(axis=0)) / chain_len
/Users/venv/lib/python3.6/site-packages/numpy/core/fromnumeric.py:3584: RuntimeWarning: Degrees of freedom <= 0 for slice
  **kwargs)
/Users/venv/lib/python3.6/site-packages/numpy/core/_methods.py:186: RuntimeWarning: invalid value encountered in true_divide
  arrmean, rcount, out=arrmean, casting='unsafe', subok=False)
/Users/venv/lib/python3.6/site-packages/numpy/core/_methods.py:207: RuntimeWarning: invalid value encountered in true_divide
  ret, rcount, out=ret, casting='unsafe', subok=False)
WARNING:pystan:Truncated summary with the 'fit.__repr__' method. For the full summary use 'print(fit)'

When I hit print(fit), I get
ValueError: Stan model anon_model_c827ea328eaaf47fe42cabc2262ba4a1 contains no samples.

All model worked just fine before.

4

Update 3 (feeling really stupid) - so all models crush when I run then with 1 chain and small number of iterations (500). All goes back to normal when I increase both (4 chains and 1500 iterations). Have no idea why this is, but all works again.

5

Don’t feel stupid, that’s definitely not supposed to crash. Can you show the exact code that crashes?

1 Like
6

this is one of them (the one we discussed in a separate thread)

functions {
    real comp_error(real c, real Delta, real sigma){
    real out;
    out = 0.5 * (erfc((c-Delta)/(sqrt(2)*sigma)) + erfc((c+Delta)/(sqrt(2)*sigma)));
    return out;
    }
}

data {
    int<lower=1> N;                                             
    int<lower=1> P;                                             
    int<lower=1> T_max;                                        
    int<lower=0> Tr[N];                                        
    int<lower=0> Nb[N, T_max];
    int<lower=0> Tar[N, T_max];
    real<lower=0> Delta[N, T_max];   
    int<lower=-1, upper=1> choice_cd[N, T_max];             
    }

parameters {
    // Hyper(group)-parameters
    vector[P] mu_prior;
    vector<lower=0>[P] tau;

    // Subject-level raw parameters
    vector[N] c_pr;
    vector[N] sigma_pr;  
}

transformed parameters {
    vector<lower=0>[N] c;
    vector<lower=0>[N] sigma;

    c = exp(c_pr);
    sigma = exp(sigma_pr);
}

model {
    real f;
    real h;

    //priors
    mu_prior ~ normal(0,5); //weakly informative priors on the regression coefficients
    tau ~ cauchy(0,2.5); //weakly informative priors, see section 6.9 in STAN user guide
    c_pr ~ normal(mu_prior[1],tau[1]);
    sigma_pr ~ normal(mu_prior[2],tau[2]);

    for (n in 1:N) {
        f = comp_error(c[n], 0, sigma[n]);
        for (t in 1:Tr[n]) {
                                        
            if (choice_cd[n,t] == 1 && Delta[n,t] > 0) {
                h = comp_error(c[n], Delta[n,t], sigma[n]);
                target += log(1 - (pow((1-h), Tar[n,t]) * pow((1-f),(Nb[n,t]-Tar[n,t]))));                
            }
            else if (choice_cd[n,t] == 0 && Delta[n,t] > 0) {
                h = comp_error(c[n], Delta[n,t], sigma[n]);
                target += log(pow((1-h),Tar[n,t]) * pow((1-f),(Nb[n,t]-Tar[n,t])));
            }
            else if (choice_cd[n,t] == 1 && Delta[n,t] == 0) {
                target += log(1 - pow((1-f),Nb[n,t]));
            }
            else if (choice_cd[n,t] == 0 && Delta[n,t] == 0) {
                target += log(pow((1-f),Nb[n,t]));
            }        
        }
    }
}
7

Hey, @ahartikainen, PyStan devs might want to know that

model.sampling(iter=500,warmup=500)

segfaults. That seems kind of bad.

@nerpa note that iter includes warmup. If you want 500 warmup and 500 post-warmup samples you need to call

model.sampling(iter=1000,warmup=500,chains=1)
8

yeah! Learning it the hard way. I had
fit = stan.sampling(data=data, iter=500, chains=1, warmup=500, thin=5)

Thanks again!

9

It doesn’t explain why it segfaults, but calling with sampling(iter=500, warmup=500) explains the zero division errors from Python – even if it didn’t crash there wouldn’t be any samples available.

10

Good to know. It did that some point with Windows.

I think we should add a check for that.

@ariddell I think this is already fixed in pystan3