Brms + cmdstanr + update() = "The desired updates require recompiling the model"

Interfaces CmdStan
1

Should it be possible to fit a model in brms with backend = 'cmdstanr' and update a fit object with new data without recompiling? I’m getting, “The desired updates require recompiling the model”. A user on the brms github issues page reported the same behavior about a year ago. The end of that threat points to an open issue on cmdstanr.

In his reply on the brms issue, Paul notes, “cmdstanr seems to be unable to run a model compiled in another R session”. I’m guessing this is because the original example saved the fit to a file and loaded it back in. I removed this.

Should the following work without recompiling?

library(brms)
library(cmdstanr)

data <- data.frame(y = rnorm(500))
threads <- threading(threads = 4)

compiled <- brms::brm(y ~ 1, data = data, 
                      chains = 1, iter = 100,
                      backend = 'cmdstanr')

newdata <- data.frame(y = rnorm(500, mean = 10))
fit <- update(compiled, newdata = newdata, 
              chains = 1, iter = 1000, warmup = 500)

Session

R version 4.0.2 (2020-06-22)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] bayestestR_0.11.5   sjPlot_2.8.9        tidybayes_3.0.2    
 [4] cmdstanr_0.2.2      brms_2.16.3         Rcpp_1.0.8
2

Hi @Eric_Green , can you try the current brms version from GitHub? We recently found a bug in the code that determined if an update required recompilation and fixed it.
Your code shouldn’t suffer from the problem of moving exec files between sessions from what I can see.

1 Like
3

Thanks @scholz. I get the same result with 2.16.11.

Could you say more about moving exec files between sessions?

4

Just to make sure, did you update to the most recent cran version or actually grab the current github master branch via remotes::install_github("paul-buerkner/brms")
Alternatively, you can check the output of brms:::needs_recompilation (without the ()) for the line out <- !is.character(exe_file) || !file.exists(exe_file) If it just says !exists(exe_file) than you don’t have the fix yet.

The problem with the sessions is that the executable model files are stored in a tmp folder that gets cleared when the rsession is closed. You can get the folder via attributes(compiled$fit)$CmdStanModel$exe_file() and see if the file actually exists. If not, it has to be recompiled.

5

You can also set/change the folder where cmdstanr stores executable, so that they are not destroyed between sessions.

Use the cmdstanr’s global option cmdstanr_write_stan_file_dir and set it to some non-tmp folder.

6

@scholz I updated via github and see the line:

else if (backend == "cmdstanr") {
        exe_file <- attributes(x$fit)$CmdStanModel$exe_file()
        out <- !is.character(exe_file) || !exists(exe_file)
    }

@rok_cesnovar adding a global option worked!

options("cmdstanr_write_stan_file_dir" = "cmdstanr")
7

Did the update complete successfully? Because the code you get is not the current github version.

8

Yes, it seems to work fine.

Do you mean that this is not the correct version?

function (x) 
{
    stopifnot(is.brmsfit(x))
    backend <- x$backend %||% "rstan"
    if (backend == "rstan") {
        out <- FALSE
    }
    else if (backend == "cmdstanr") {
        exe_file <- attributes(x$fit)$CmdStanModel$exe_file()
        out <- !is.character(exe_file) || !file.exists(exe_file)
    }
    else if (backend == "mock") {
        out <- FALSE
    }
    out
}
9

Now it is. Might be that you didn’t reload the package when you posted earlier?

Could you try if the problem is fixed without the options("cmdstanr_write_stan_file_dir" = "cmdstanr") option now? If not, there would be something else in brms we’d have to look for.

10

I started a new session and ran without options("cmdstanr_write_stan_file_dir" = "cmdstanr"). I did not get the note about recompiling. Seems like the update did the trick.

1 Like
11

Great!

@scholz so if I am getting this correctly, there is no need for Add a way to optionally store exe in object · Issue #512 · stan-dev/cmdstanr · GitHub for brms anymore?

12

There is still the problem that the exe files are removed when the session is closed. Which is a problem I ran into when using cmdstan on my local cluster. Even without closing the cluster. So it would still be cool to be able to store those files somewhere permanent.
See here for a small summary of the problem I had.

1 Like