Controlling Stan_glm output

Interfaces rstanarm
1

I am confused as to how to minimize output from StanArm in an Rmarkdown file. I am using Regression and Other Stories to teach and reworking examples I have used in the past and ideally I want to not see all the output I see below. I did first search documentation before I posted here. I am a beginner using Stan_glm, (one could argue given my increased workload and family load given the pandemic this was not the time to learn while also teaching, but I was so excited about the new book I wanted to teach with it) . Thank you for your help. I wonder if there is something small I am missing about controlling output.

Student Only Model Using Stan GLM Defaults

lm_stan_default <- stan_glm(Balance ~ Student, data = Credit)

## 
## SAMPLING FOR MODEL 'continuous' NOW (CHAIN 1).
## Chain 1: 
## Chain 1: Gradient evaluation took 2.3e-05 seconds
## Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 0.23 seconds.
## Chain 1: Adjust your expectations accordingly!
## Chain 1: 
## Chain 1: 
## Chain 1: Iteration:    1 / 2000 [  0%]  (Warmup)
## Chain 1: Iteration:  200 / 2000 [ 10%]  (Warmup)
## Chain 1: Iteration:  400 / 2000 [ 20%]  (Warmup)
## Chain 1: Iteration:  600 / 2000 [ 30%]  (Warmup)
## Chain 1: Iteration:  800 / 2000 [ 40%]  (Warmup)
## Chain 1: Iteration: 1000 / 2000 [ 50%]  (Warmup)
## Chain 1: Iteration: 1001 / 2000 [ 50%]  (Sampling)
## Chain 1: Iteration: 1200 / 2000 [ 60%]  (Sampling)
## Chain 1: Iteration: 1400 / 2000 [ 70%]  (Sampling)
## Chain 1: Iteration: 1600 / 2000 [ 80%]  (Sampling)
## Chain 1: Iteration: 1800 / 2000 [ 90%]  (Sampling)
## Chain 1: Iteration: 2000 / 2000 [100%]  (Sampling)
2

Hi @LauraK, good question! I think this is an example of missing information in our documentation. You should be able to set refresh=0 as an additional argument to avoid printing the iteration updates. Does that work for you?

I just opened an issue in the rstanarm github repository about this so that we remember to document it better going forward:

1 Like
3

that does work!
Thank you.
I agree that it would be great if this was better documented. I might even consider making this the default.

1 Like
4

Great, glad that works!

Yeah, I think that’s worth considering. Especially for models that run pretty quickly I think you’re right that it makes sense to suppress the iteration output by default. For slower models (and with MCMC there are lots of slower models unfortunately!) it’s nice to see updates so you can check the progress and know that it’s running and not just silently stuck. If the default is not to show the updates then I think what could end up happening a lot is that people with slow models would start running them and then wonder if it’s working properly (because it’s just silent but taking a long time), but they would need to cancel their progress and rerun their model in order to turn on the progress updates, which would be annoying. If we could automatically estimate how long a model would take to run then I guess we could have a default that depends on that, but that can be pretty hard. Anyway, I think there are arguments both ways. At a minimum it needs to be documented better, but we should also revisit the issue of what the default should be. Thanks a lot for the feedback, I really appreciate it!

5

I’m actually having the opposite issue. I would like to see the iteration updates but since updating R and rstanarm they have been supressed for me. When checking old and new rstanarm and rstan documentation and source code it appears there has been no change to the deafult. Or have I missed something? refresh = 0 should supress them and refresh = >0 should show them (or refresh = NULL). I have run the model with these options direct in the R console and through Rmarkdown in RStudio and still I see no iteration updates. Thanks in advance.

rstanarm 2.21.1
R Studio 1.3.1093
R 4.0.3 (2020-10-10)
Running under: macOS Catalina 10.15.6

6

Hi Rachel, you don’t have anything left in one of these files that might prohibit the output? Never heard about this problem before :)

~/.Rprofile
~/.Renviron
7

Hey Rachel, you’re right and you didn’t miss anything. This is definitely strange! I have the same exact versions of everything as you do (except I have Catalina 10.15.7 instead of 10.15.6), and refresh works as expected for me.

In addition to checking the files @torkar asked about, can you try this (seems like a lot of steps but they’re super quick):

  1. Close all R sessions you have open
  2. Start a fresh R session in RStudio or in the terminal
  3. Run remove.packages(c("rstanarm", "rstan")).
  4. Run install.packages("rstanarm") and if it asks you if you want to install rstan from source say no
  5. In RStudio (but not inside RMarkdown) or in the terminal try
library(rstanarm)
fit <- stan_glm(mpg ~ wt, data = mtcars)

and let us know if you still don’t see any printed output.

Thanks and sorry for the inconvenience!

8

Thank you @torkar and @jonah for taking the time to help me. I’ve actually worked out the culprit! When I run chains in parallel

library(rstanarm)
options(mc.cores = parallel::detectCores())
fit <- stan_glm(mpg ~ wt, data = mtcars)

(which I almost always do to make things quicker) this supresses the iteration updates for this version of R and rstanarm (at least for me).

When I run chains one by one (the default)

library(rstanarm)
fit <- stan_glm(mpg ~ wt, data = mtcars)

I see the iteration updates (hurah!). Do you have the same issue?

I still checked .Rprofile or .Renviron just incase and all looks fine (I don’t touch them normally). Plus I re-installed rstanarm as you instructed @jonah These don’t make a difference.

I remember having issues when I first updated to R v. 4.0.0 with running chains in parallel (from memory this was a Catalina issue) that was resolved with the next update of R.

I’ll look up the source code of the parallel function and documentation to see if I can understand what’s going on.

9

Happy to help. Thanks for letting us know about this issue.

I see the iteration updates regardless of whether I’m running in parallel or not. I wish I knew why it’s working ok for me but not for you but I’m currently stumped.

Thanks for trying that.

10

Actually I think I figured out what’s going on. Right after I responded I realized I forgot to check my version of rstan. I currently have the development version (not the CRAN version) of rstan installed. The printed iteration updates from rstanarm are coming internally from rstan so it’s really the version of rstan that matter for this. I just switched back to the CRAN version of rstan as a test and now I’m seeing the same problem that you are with rstanarm in parallel.

So I think this should be fixed by the next release of rstan, although I’m still not sure what the actual cause of the problem is.

If you don’t want to wait until then (I don’t know when that will be yet but probably sometime this winter) then you can try installing the development version of rstan from github with

# install 'remotes' package first if necessary
remotes::install_github("stan-dev/rstan", ref = "develop", subdir = "rstan/rstan")

Unfortunately, unlike installing the rstan binary from CRAN, installing rstan from source via github requires compiling the c++ code in rstan. If you don’t already have the c++ toolchain set up then you can use these instructions before trying to install rstan:

If you try that let us know if it solves the problem with rstanarm or if you run into trouble installing rstan.

1 Like
11

Thanks @jonah I tried installing the development version of rstan but I’m having issues with the Rtools for Mac. I believe this may be because I don’t have full admin access on this laptop (university owned) and it’s preventing the software to run after installed. I was able to to install the development version of Rstan on my friend’s computer (but it’s with Linux) and everything worked fine, including the iteration updates.

I might just wait. Thanks for your patience!

2 Likes
12

Yeah that’s possible. There may be some way around that but dealing with permissions and admin access issues is not my area of expertise.

Ok well that’s a good sign, thanks for trying that.

Thanks for letting us know about this!

1 Like