# Cmdstanr::cmdstan\_model can compile model, but fails to make it available in R, unless already compiled

**URL:** <https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939>\
**Category:** CmdStan\
**Tags:** cmdstanr\
**Created:** [September 30, 2023, 8:50am UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939 "2023-09-30T08:50:25Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![adamgorm](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/adamgorm/32/19161_2.png) [@adamgorm](https://discourse.mc-stan.org/u/adamgorm)\
**Post date:** [September 30, 2023, 8:50am UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/1 "2023-09-30T08:50:25Z")

</div>

When I call `cmdstan_model`, it can compile and save the model, but it gives the error `Error in file(con, "r") : cannot open the connection` when trying to actually make it available in R. However, if the model is already compiled, then it correctly returns a CmdStanModel.

# Example

The following code fails with the error `Error in file(con, "r") : cannot open the connection`:

```no-highlight
library(cmdstanr)
file <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
mod <- cmdstan_model(file)

```

When running with `options("cmdstanr_verbose" = TRUE)` the final part (about linking the model) is:

```no-highlight
--- Linking model ---
g++ -std=c++1y -pthread -D_REENTRANT -Wno-sign-compare -Wno-ignored-attributes -I stan/lib/stan_math/lib/tbb_2020.3/include -O3 -I src -I stan/src -I stan/lib/rapidjson_1.1.0/ -I lib/CLI11-1.9.1/ -I stan/lib/stan_math/ -I stan/lib/stan_math/lib/eigen_3.4.0 -I stan/lib/stan_math/lib/boost_1.78.0 -I stan/lib/stan_math/lib/sundials_6.1.1/include -I stan/lib/stan_math/lib/sundials_6.1.1/src/sundials -DBOOST_DISABLE_ASSERTS -Wl,-L,"/home/adam/.local/opt/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/home/adam/.local/opt/cmdstan/stan/lib/stan_math/lib/tbb" /tmp/RtmpZPELHp/model-463e2ffb1b99.o src/cmdstan/main.o -Wl,-L,"/home/adam/.local/opt/cmdstan/stan/lib/stan_math/lib/tbb" -Wl,-rpath,"/home/adam/.local/opt/cmdstan/stan/lib/stan_math/lib/tbb" stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_nvecserial.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_cvodes.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_idas.a stan/lib/stan_math/lib/sundials_6.1.1/lib/libsundials_kinsol.a stan/lib/stan_math/lib/tbb/libtbb.so.2 -o /tmp/RtmpZPELHp/model-463e2ffb1b99
rm /tmp/RtmpZPELHp/model-463e2ffb1b99.hpp /tmp/RtmpZPELHp/model-463e2ffb1b99.o
Error in file(con, "r") : cannot open the connection

```

However, if I rerun the code (so the compiled model is already saved), it works.

# Setup

Operating system: Ubuntu.

```no-highlight
> library(cmdstanr)
This is cmdstanr version 0.6.1.9000
- CmdStanR documentation and vignettes: mc-stan.org/cmdstanr
- CmdStan path: /home/adam/.local/opt/cmdstan
- CmdStan version: 2.33.1
> check_cmdstan_toolchain()
The C++ toolchain required for CmdStan is setup properly!
> cmdstan_path()
[1] "/home/adam/.local/opt/cmdstan"
> cmdstan_version()
[1] "2.33.1"
> tempdir()
[1] "/tmp/Rtmp6MIhLF"

```

The problem appeared after I updated cmdstan to version 2.33.1 (from version 2.32, I think) and cmdstanr to the newest version from GitHub (I am not sure what version I had before).

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [September 30, 2023, 11:04am UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/2 "2023-09-30T11:04:28Z")

</div>

This could be a tricky one to debug. Our CI runs this exact combination of OS/cmdstan/cmdstanr without error, so it looks like there is some local config causing issues.

If you run `traceback()` after getting that error, does it report anything?

If you reinstall cmdstan (i.e., `reinstall_cmdstan(cores=...)`) does the error persist?

---

<div class="post-metadata">

**Author:** ![adamgorm](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/adamgorm/32/19161_2.png) [@adamgorm](https://discourse.mc-stan.org/u/adamgorm)\
**Post date:** [September 30, 2023, 11:38am UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/3 "2023-09-30T11:38:28Z")

</div>

I just tried calling `rebuild_cmdstan()` and the problem persists.

`traceback()` doesn’t seem to give anything very interesting (see below). I am considering whether my best bet is to wipe my entire R installation (incl. all packages) and hope that the problem somehow fixes itself by doing a complete reinstall/reconfig.

Result from `traceback()` after error:

```plaintext
8: file(con, "r")
7: readLines(private$hpp_file_, warn = FALSE)
6: withCallingHandlers(expr, warning = function(w) if (inherits(w, 
       classes)) tryInvokeRestart("muffleWarning"))
5: suppressWarnings(private$model_methods_env_$hpp_code_ <- readLines(private$hpp_file_, 
       warn = FALSE))
4: self$compile(...)
3: initialize(...)
2: CmdStanModel$new(stan_file = stan_file, exe_file = exe_file, 
       compile = compile, ...)
1: cmdstan_model(file)

```

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [September 30, 2023, 11:57am UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/4 "2023-09-30T11:57:37Z")

</div>

That output is actually quite helpful. It’s saying that the `.hpp` generated by `cmdstan` from your Stan code is no longer in the temporary directory. Your system might be very aggressively cleaning out the tempdir for some reason.

Can you try calling `cmdstan_model()` with the `output_dir`argument set to some local directory that you can see? We need to verify whether the error occurs while the`.hpp` file is still present

---

<div class="post-metadata">

**Author:** ![adamgorm](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/adamgorm/32/19161_2.png) [@adamgorm](https://discourse.mc-stan.org/u/adamgorm)\
**Post date:** [October 1, 2023, 7:01am UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/5 "2023-10-01T07:01:04Z")

</div>

When running `mod <- cmdstan_model(file, dir = "/home/adam/cmdstanr_test")` I still get the same error and the traceback

```plaintext
8: file(con, "r")
7: readLines(private$hpp_file_, warn = FALSE)
6: withCallingHandlers(expr, warning = function(w) if (inherits(w, 
       classes)) tryInvokeRestart("muffleWarning"))
5: suppressWarnings(private$model_methods_env_$hpp_code_ <- readLines(private$hpp_file_, 
       warn = FALSE))
4: self$compile(...)
3: initialize(...)
2: CmdStanModel$new(stan_file = stan_file, exe_file = exe_file, 
       compile = compile, ...)
1: cmdstan_model(file, dir = "/home/adam/cmdstanr_test")

```

As before, the executable `bernoulli` does appear in the `/home/adam/cmdstanr_test` dir and if I rerun `mod <- cmdstan_model(file, dir = "/home/adam/cmdstanr_test")` it correctly loads it with the message `Model executable is up to date!`.

I just tried running the `cmdstan_model` command while keeping an eye on the dir given by `tempdir()` (which happens to be `/tmp/RtmpncngT8`).

- In the beginning it contains `releases-213c3727b5b0.json`.
- While running `cmdstan_model` compilation it contains `model-213c4019897c.stan` and `model-213c4019897c.hpp`.
- After finishing the `cmdstan_model` (with throwing the error) the .hpp file disappears and the executable `model-213c4019897c` appears.

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 1:34pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/6 "2023-10-01T13:34:16Z")

</div>

Ah hold on, I think I see where this is coming from. The verbose output you posted above includes a step which is deleting the `.hpp` file:

```plaintext
rm /tmp/RtmpZPELHp/model-463e2ffb1b99.hpp /tmp/RtmpZPELHp/model-463e2ffb1b99.o

```

This is occuring as part of the initial model compilation, so the `hpp` is deleted before R has a chance to get to it.

This isn’t a default step in the make command, and doesn’t occur for me (@WardBrian do you know if this has changed recently or similar?).

Do you have any custom CmdStan flags/similar set? What’s your output from `cmdstan_make_local()`?

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 1:35pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/7 "2023-10-01T13:35:50Z")

</div>

But this does appear to be a bit fragile, so I’ll update the `cmdstanr` process to generate the `.hpp` file again if it’s not present

---

<div class="post-metadata">

**Author:** ![WardBrian](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/wardbrian/32/12078_2.png) [@WardBrian](https://discourse.mc-stan.org/u/WardBrian)\
**Post date:** [October 1, 2023, 1:50pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/8 "2023-10-01T13:50:00Z")

</div>

> [@andrjohns](#):
>
> do you know if this has changed recently

Yes, recently we un-marked the .hpp as .PRECIOUS so the makefile won’t keep it around unless you ask for it

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 1:56pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/9 "2023-10-01T13:56:24Z")

</div>

Ah got it, makes sense! I’ll update the `cmdstanr` behaviour to address this, thanks!

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 2:01pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/10 "2023-10-01T14:01:45Z")

</div>

@adamgorm Just to double-check, are you using the GitHub/`develop` version of `cmdstan`? It looks like this change isn’t present in the 2.33.1 release, only in the current develop branch

---

<div class="post-metadata">

**Author:** ![adamgorm](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/adamgorm/32/19161_2.png) [@adamgorm](https://discourse.mc-stan.org/u/adamgorm)\
**Post date:** [October 1, 2023, 2:05pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/11 "2023-10-01T14:05:47Z")

</div>

Yes, I installed the newest `cmdstanr` yesterday by running `remotes::install_github("stan-dev/cmdstanr")`.

Edit: Ah, you asked about `cmdstan`. There I use the 2.33.1 release.

New edit: I am actually also using the development version of `cmdstan`.

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 2:06pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/12 "2023-10-01T14:06:34Z")

</div>

I mean `cmdstan` itself (not the R package)

---

<div class="post-metadata">

**Author:** ![adamgorm](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/adamgorm/32/19161_2.png) [@adamgorm](https://discourse.mc-stan.org/u/adamgorm)\
**Post date:** [October 1, 2023, 2:09pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/13 "2023-10-01T14:09:04Z")

</div>

No, wait, I am actually using the development version of `cmdstan` as well, I think. I reinstalled yesterday by running `> git clone https://github.com/stan-dev/cmdstan.git --recursive` as described in the manual.

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 2:12pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/14 "2023-10-01T14:12:01Z")

</div>

Yep, that’ll do it. I’ll update `cmdstanr` to handle the new changes, but if you want things to work in the interim then you can revert to the release version by deleting that folder calling `install_cmdstan()` again

---

<div class="post-metadata">

**Author:** ![adamgorm](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/adamgorm/32/19161_2.png) [@adamgorm](https://discourse.mc-stan.org/u/adamgorm)\
**Post date:** [October 1, 2023, 2:24pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/15 "2023-10-01T14:24:29Z")

</div>

Yes, you’re right! I just deleted my `cmdstan` installation and reinstalled with `install_cmdstan()` and now it works flawlessly.

Thanks a lot for the help and for the very fast replies!

---

<div class="post-metadata">

**Author:** ![andrjohns](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/andrjohns/32/15297_2.png) [@andrjohns](https://discourse.mc-stan.org/u/andrjohns)\
**Post date:** [October 1, 2023, 2:30pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/16 "2023-10-01T14:30:15Z")

</div>

No worries, thanks for flagging this! It’s very helpful to know about it before the next release

---

<div class="post-metadata">

**Author:** ![nicholasjclark](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/nicholasjclark/32/14962_2.png) [@nicholasjclark](https://discourse.mc-stan.org/u/nicholasjclark)\
**Post date:** [January 18, 2024, 10:19pm UTC](https://discourse.mc-stan.org/t/cmdstanr-cmdstan-model-can-compile-model-but-fails-to-make-it-available-in-r-unless-already-compiled/32939/17 "2024-01-18T22:19:00Z")

</div>

Hi all, just a note that I’m now getting this same error on the latest version of Cmdstan. My setup is:

```no-highlight
> version
               _                                
platform x86_64-w64-mingw32               
arch x86_64                           
os mingw32                          
crt ucrt                             
system x86_64, mingw32                  
status                                          
major 4                                
minor 3.1                              
year 2023                             
month 06                               
day 16                               
svn rev 84548                            
language R                                
version.string R version 4.3.1 (2023-06-16 ucrt)
nickname Beagle Scouts  

> cmdstan_version()
[1] "2.34.0"

> utils::packageVersion('cmdstanr')
[1] ‘0.7.0’

```

The traceback looks like this:

```no-highlight
Error in file(con, "r") : cannot open the connection
9. file(con, "r")
8. readLines(private$hpp_file_, warn = FALSE)
7. withCallingHandlers(expr, warning = function(w) if (inherits(w, classes)) tryInvokeRestart("muffleWarning"))
6. suppressWarnings(private$model_methods_env_$hpp_code_ <- readLines(private$hpp_file_,
warn = FALSE))
5. self$compile(...)
4. initialize(...)
3. CmdStanModel$new(stan_file = stan_file, exe_file = exe_file,
compile = compile, ...)
2. cmdstanr::cmdstan_model(cmdstanr::write_stan_file(vectorised$model_file)) at mvgam.R#1908

```

My make file is:

```no-highlight
> cmdstan_make_local()
[1] "CXXFLAGS += -Wno-nonnull -D_UCRT"        
[2] "TBB_CXXFLAGS= -D_UCRT"                   
[3] "CXXFLAGS += -Wno-deprecated-declarations"
[4] "CXXFLAGS += -Wno-nonnull -D_UCRT"        
[5] "TBB_CXXFLAGS= -D_UCRT"                   
[6] "CXXFLAGS += -Wno-deprecated-declarations"

```

Any help would be greatly appreciated

EDIT: I’ve just upgraded to the latest development version of `cmdstanr` (0.7.1) and everything seems to be working now. I’ll leave this post here though in case anyone else runs into similar troubles