Brms and cmdstanr show un-hidable warnings about proposal rejections

andrewheiss · October 10, 2023, 2:53am

On macOS Sonoma with brms 2.20.5, cmdstanr 0.6.1 and cmdstan 2.33.1 I’m getting 2–8ish warnings about proposal rejections that appear regardless of silent or refresh settings (i.e. even when using silent = 2, refresh = 0.

These warnings didn’t appear with earlier versions of brms and/or cmdstanr—I’m not sure which version of which package introduced them (there’s an issue at {cmdstanr} from April).

The model fits fine; it just emits a ton of unhidable output now.

> brms::brm(mpg ~ cyl, data = mtcars, backend = "cmdstanr", silent = 2)
Compiling Stan program...
ld: warning: duplicate -rpath '/Users/andrew/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb' ignored

Start sampling
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 1 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 1 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 1 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 1 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 1 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 2 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 2 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 2 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 2 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 2 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 3 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 3 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 3 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 3 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 3 
Chain 4 Informational Message: The current Metropolis proposal is about to be rejected because of the following issue:
Chain 4 Exception: normal_id_glm_lpdf: Scale vector is inf, but must be positive finite! (in '/var/folders/17/g3pw3lvj2h30gwm67tbtx98c0000gn/T/RtmpUngC6I/model-ca65460d50d3.stan', line 34, column 4 to column 62)
Chain 4 If this warning occurs sporadically, such as for highly constrained variable types like covariance matrices, then the sampler is fine,
Chain 4 but if this warning occurs often then your model may be either severely ill-conditioned or misspecified.
Chain 4 
 Family: gaussian 
  Links: mu = identity; sigma = identity 
Formula: mpg ~ cyl 
   Data: mtcars (Number of observations: 32) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Population-Level Effects: 
          Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
Intercept    37.89      2.18    33.56    42.15 1.00     3126     2713
cyl          -2.88      0.34    -3.54    -2.19 1.00     3084     2590

Family Specific Parameters: 
      Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
sigma     3.33      0.45     2.59     4.33 1.00     3142     2445

Draws were sampled using sample(hmc). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

andrewheiss · October 10, 2023, 4:37am

I’m fairly certain it’s a {brms} related issue. When running using {cmdstanr} with raw Stan code, there are no warnings or messages (beyond a duplicate -rpath warning related to this Warning of duplicate -rpath when compiling and installing CmdStan) :

file <- file.path(cmdstan_path(), "examples", "bernoulli", "bernoulli.stan")
mod <- cmdstan_model(file)
#> Compiling Stan program...
#> ld: warning: duplicate -rpath '/Users/andrew/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb' ignored

This post here ( Diagnosing machine-specific sampling problem (brms, cmdstanr)) seems to have a similar (identical?) issue—I haven’t tried any more complex models yet to see if things sample.

This post by @bbolker seems to be the same/similar issue too Informational messages (normal_id_glm_lpdf: Scale vector is inf, but must be positive finite)

jonah · October 10, 2023, 8:47pm

Some changes to how cmdstanr handles Stan’s messages and warnings unfortunately didn’t play nicely with the way the silent argument works in brms:

github.com/paul-buerkner/brms

Update translation of silent argument to fix cmdstanr breakage

paul-buerkner:master ← andrjohns:fix-cmdstanr-silent

opened 10:00AM - 10 Oct 23 UTC

andrjohns

+2 -1

Updates the `silent` -> `show_messages`,`show_exceptions` translation for `cmdst…anr` to the following mapping: - `silent=0` suppress nothing (`show_messages=TRUE, show_exceptions=TRUE`) - `silent=1` suppress only exceptions (`show_messages=TRUE, show_exceptions=FALSE`) - `silent=2` suppress messages and exceptions (`show_messages=FALSE, show_exceptions = FALSE`)

andrewheiss · October 10, 2023, 10:29pm

Oh awesome, that fixes it!

> brms::brm(mpg ~ cyl, data = mtcars, backend = "cmdstanr")
Compiling Stan program...
ld: warning: duplicate -rpath '/Users/andrew/.cmdstan/cmdstan-2.33.1/stan/lib/stan_math/lib/tbb' ignored

Start sampling
Running MCMC with 4 sequential chains...

Chain 1 Iteration:    1 / 2000 [  0%]  (Warmup) 
Chain 1 Iteration:  100 / 2000 [  5%]  (Warmup) 
Chain 1 Iteration:  200 / 2000 [ 10%]  (Warmup) 
...
All 4 chains finished successfully.
Mean chain execution time: 0.0 seconds.
Total execution time: 0.6 seconds.

jonah · October 11, 2023, 12:07am

Awesome. Thanks for testing it.

And thanks @andrjohns for handling the PR.

blokeman · October 13, 2023, 9:06pm

How do I install this update, please?

I already tried reinstalling the latest development version with remotes::install_github("stan-dev/cmdstanr"), but I still can’t see the sampling process.

jonah · October 13, 2023, 9:53pm

The relevant update was made in brms to adjust to the change in cmdstanr: Update translation of silent argument to fix cmdstanr breakage by andrjohns · Pull Request #1548 · paul-buerkner/brms · GitHub

So you’d need to install the brms package from GitHub:

remotes::install_github("paul-buerkner/brms")

Does that work?

blokeman · October 16, 2023, 6:59am

After updating brms as instructed, models won’t sample at all. There’s the following error:

Error in '/tmp/Rtmpt10WNx/model-b554b11a77561.stan', line 101, column 2: Declaration
    of arrays by placing brackets after a variable name was removed in Stan
    2.33.0. Instead use the array keyword before the type. This can be
    changed automatically using the auto-format flag to stanc

I’m trying to fix it on my own, but haven’t yet found how to change that setting globally for the whole session.

When updating brms, it also offered to update a bunch of other libraries related to Stan, so apparently one of those updates broke something. RStudio now lists brms version as 2.20.5, cmdstan version as 2.33.1, cmdstanr version as 0.6.1.90000, along with both rstan and StanHeaders versions as 2.32.2. That cmdstanr version looks very suspicious to me.

UPDATE: I have downgraded to brms version 2.19.0 for now. Inability to see the sampling process is a minor annoyance compared to the inability to fit models at all.

Perhaps these complaints of mine ought to be moved to the unresolved thread. The unhideable warnings do not bother me, I’d just appreciate being able to see how far the chains have gotten…

tjmahr · October 17, 2023, 2:12pm

The cmdstanr version conveys with the .9000 that it’s a development version of a package rather than a proper release version.

blokeman · October 17, 2023, 8:48pm

Thanks. Anyway, I did try downgrading CmdStanR as well, to no avail. @andrewheiss what versions are you running, that the fix worked?

andrewheiss · October 17, 2023, 9:15pm

On macOS Sonoma 14.0 with R 4.3.1:

cmdstan 2.33.1
cmdstanr 0.6.1
rstan 2.32.3
StanHeaders 2.26.28
brms 2.20.5

I have this in ~/.cmdstan/cmdstan-2.33.1/make/local (via Cannot use method suggested by Rok to remove excessive sampler warnings · Issue #805 · stan-dev/cmdstanr · GitHub), but I think this just hides a bunch of warnings:

CXXFLAGS += -Wno-nonnull -D_UCRT -Wno-deprecated-declarations -Wno-unused-lambda-capture -Wno-unused-variable -Wno-unknown-pragmas
TBB_CXXFLAGS= -D_UCRT -Wno-unused-but-set-variable
CXXFLAGS_BOOST += -Wno-deprecated-declarations

JLC · October 24, 2023, 5:51pm

I have the same issue and also had to downgrade brms to 2.19.0 to keep things functional.

Topic		Replies	Views
Diagnosing machine-specific sampling problem (brms, cmdstanr) brms cmdstanr , brms	0	260	August 25, 2023
How to suppress compiler warnings from brms with cmdstanr backend? brms compiler	4	417	November 9, 2023
Problematic sampiling with brms after updating cmdstan brms cmdstan	9	372	September 27, 2023
Cmdstan/brms/cmdstanr ignores silent and refresh arguments, won't show sampling process brms cmdstan , cmdstanr	1	377	October 7, 2023
Cmdstanr / brms issues error messages , after update brms brms	2	409	July 2, 2023

Brms and cmdstanr show un-hidable warnings about proposal rejections

Related Topics