Stan_surv crashes R repeatedly

Interfaces rstanarm
1

I am trying to fit a model using stan_surv from the rstanarm package. The crash happens when I try to fit this model:

fit <- stan_surv(Surv(time, event) ~ 1, data = df, basehaz = 'exp')

The model (and any other stan_surv model that I have tried) immediate sends an error that says

====== C stack trace ==========
RaiseException [...

and the R crashes. R and all packages are up-to-date. System information:

R version 4.0.3 (2020-10-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
AMD Ryzen 9 9350X 16-core processor, 64 GB RAM
Running under: Windows 10 x64 (build 19041)
rstanarm_2.21.2
rstan_2.21.2

Interestingly, this crash occurs on my desktop PC (my supposed power computer for running Bayesian things) and did not occur on my MS SurfacePro laptop last night. Both are running what I believe to be the same versions of rstan and rstanarm.

I have no idea why this happening. Everything else in R seems to be working, including accessing stan via survHE. So I suspect this is an rstanarm problem. No idea how to check. Help appreciated.

Brant

2

Update: I completely removed R, Rstudio, Rtools, and all packages and re-installed. Problem still persists. Reproducible using standard datasets including pbc from the survival package.

3

Sorry for the troubles. How did you install the survival branch?

4

Rok,

Thank you for your reply. Always appreciate when someone takes time to help. I installed rstanarm using this command.

install.packages(“rstanarm”, repos = c(“https://mc-stan.org/r-packages/”, getOption(“repos”)))

Prior to that, I followed the instructions on Stan installation here:

https://github.com/stan-dev/rstan/wiki/RStan-Getting-Started

As stated In post, I have removed and reinstalled the rstanarm package several times, and did a complete R delete and reinstall too, both to no avail. I plan on testing non-survival models with rstanarm tomorrow to see if it is the entire package or just the survival function that is causing the crash. Running Stan models with survHE does not crash R, so it does not appear to be a Stan problem per se.

Also, my PC is dual booted with Linux on a second M.2 harddrive, so I will try Linux tomorrow to ensure it is not some odd hardware issue.

Brant

6

Update: I can reproduce the problem on Linux and Windows using rstan and rstanarm 2.21.2.

library(survival)
library(rstan)
library(rstanarm)
data(pbc)

# Make a new outcome that is binary
pbc$event <- ifelse(pbc$status == 2, 1, 0)

# Make a fake clustering variable
pbc$hospital <- ifelse(rbinom(nrow(pbc), 1, 0.5) > 0.5, 'A', 'B')

#Fit a frailty model:
fit1 <- stan_surv(Surv(time, event) ~ trt + (1 | hospital), basehaz = 'ms', data = pbc)

#Fit same model without frailties
fit2 <- stan_surv(Surv(time, event) ~ trt, basehaz = 'ms', data = pbc)

Fit 1 works. Fit 2 crashes R every time. It crashes for all the baseline hazard distributions (exp, gompertz, weibull, ms, and bs).

I noticed that trt had missing values. So I tried age which does not have any missing values.

fit3 <- stan_surv(Surv(time, event) ~ age + (1 | hospital), basehaz = 'ms', data = pbc)
fit4 <- stan_surv(Surv(time, event) ~ age, basehaz = 'ms', data = pbc)
fit5 <- stan_surv(Surv(time, event) ~ 1, basehaz = 'ms', data = pbc)

Fit 3 works. Fits 4 and 5 crash R. I conclude that something about how stan_surv is handling covariates is to blame. Unfortunately, my technical ability is limited and this is as far as I can point out the issue. Would appreciate insight.

Brant

1 Like
7

This is a bit out of my scope as well. Paging @sambrilleman :)

8

I can reproduce the crash in 2.21 but it works fine in the experimental 2.26, interestingly enough.

@binman if you want to give the experimental version of RStan a try (with the caveat that it is still experimental), you can install it via:

remove.packages(c("StanHeaders", "rstan","rstanarm"))
install.packages("StanHeaders", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
install.packages("rstanarm", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
9

@andrjohns I have no experience with experimental versions of Rstan (except I suppose rstanarm with stan_surv now). Is there some advice about potential bad effects that I should be aware of?

10

No, just that there might be some unexpected bugs or errors. It’s relatively stable at the moment, but even if there are issues that’s really helpful for us to know and be able to fix before it’s released to CRAN

11

First bug found:

library(survival)
library(rstan)
library(rstanarm)
data("kidney")

ms <- stan_surv(Surv(time, status) ~ age + sex + disease + (1|id), data = kidney, basehaz = 'ms')
bs <- update(ms, basehaz = 'bs')

This immediately spits out pages of error messages. They look like this

`Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4: Rejecting initial value:
Chain 4: Error evaluating the log probability at the initial value.
Chain 4: Exception: lb_constrain: lb is -inf, but must be finite! (in ‘surv’, line 476, column 4 to column 14)
Chain 4:
Chain 4: Initialization between (-2, 2) failed after 100 attempts.
Chain 4: Try specifying initial values, reducing ranges of constrained values, or reparameterizing the model.
[1] “Error in sampler$call_sampler(args_list[[i]]) : Initialization failed.”
error occurred during calling the sampler; sampling not done
here are whatever error messages were returned
[[1]]
Stan model ‘surv’ does not contain samples.

[[2]]
Stan model ‘surv’ does not contain samples.

[[3]]
Stan model ‘surv’ does not contain samples.

[[4]]
Stan model ‘surv’ does not contain samples.

Error in check_stanfit(stanfit) :
Invalid stanfit object produced please report bug
In addition: Warning messages:
1: In for (v in val) { : closing unused connection 6 (<-localhost:11173)
2: In for (v in val) { : closing unused connection 5 (<-localhost:11173)
3: In for (v in val) { : closing unused connection 4 (<-localhost:11173)
4: In for (v in val) { : closing unused connection 3 (<-localhost:11173)
5: In .local(object, …) :
some chains had errors; consider specifying chains = 1 to debug

12

Took me a minute, but figured this one out. stan_surv uses different lower limits for some parameters depending on the baseline hazard type. When the bs type is requested, those lower limits are set to negative_infinity() (i.e., no restriction). However, Stan was recently updated to check that lower and upper limits for parameter constraints were finite, causing these errors. Only the piecewise and bs hazard types are affected.

I’ll raise a discussion with the other devs about how to approach this. This is very helpful to know, but sorry that it’s getting in your way!

2 Likes
13

You have been incredibly helpful. I really appreciate it.

14

Hi, sorry I am late to this party.

Thanks @andrjohns for digging into this!! That is epic that you were able to locate the issue! I almost certainly would not have had time to dig into the details.

I imagine that this is going to continue to be a headache, as we have a static binary that won’t stay in line with dependencies. But if we can get automated tests running on the feature/survival branch (I think @jonah did a lot of work at getting the automated build and testing back up and running) then I guess we can at least hopefully catch it when it breaks and then potentially rebuild the binary after fixing any bugs or dependency issues. Of course this is all a somewhat undesirable maintenance burden, and it would probably be easier if/when stan_surv makes it into rstanarm proper.

15

@binman The issue was fixed

remove.packages(c("StanHeaders", "rstan"))
install.packages("StanHeaders", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))
install.packages("rstan", repos = c("https://mc-stan.org/r-packages/", getOption("repos")))

will install rstan 2.26 and should not have the problem you experienced.

2 Likes