Low BFMI warning with E-BFMI > 0.3

cabroncito · May 13, 2024, 1:32am

Hi, I’ve been working on a time series model with unobserved components. For now, I’ve been running things with only one chain before scaling the problem to more chains, but I have found something that is puzzling me and that is likely a result of my own ignorance.

Specifically, I get the “low BFMI” warning. As far as I understand one should get sucha a warning only if E-BFMI < 0.3. Nonetheless, when I use the get_bfmi( stan_object ) function I get that my E-BFMI = 0.71 and if I use check_energy( stan_object ) I get “E-BFMI indicated no pathological behavior”. Hence, these statistics seem to suggest that E-BMFI is not low (contradicting the warning I get). Am I interpreting something totally wrong about the “low BFMI” warning or could it be possible that there is something “funny” going on in RStan?

jsocolar · May 13, 2024, 6:25pm

Wow, it looks like the code that throws this warning is just straight-up doing the wrong thing. If true, I’m stunned that this hasn’t been noticed and fixed, since this particular bit of code has been around for a while:

github.com

stan-dev/rstan/blob/50c3c6f4e44a849b17af7e9e2b8656cd93238a18/rstan/rstan/R/check_hmc_diagnostics.R#L62


      
          if (n_m > 0)
            warning("There were ", n_m,
                    " transitions after warmup that exceeded the maximum treedepth.",
                    " Increase max_treedepth above ", max_td, ". See\n",
                    "https://mc-stan.org/misc/warnings.html#maximum-treedepth-exceeded", call. = FALSE)
          n_e <- 0L
          if (is_sfinstance_valid(object) && all(sapply(sp, function(x) "energy__" %in% colnames(x)))) {
            E <- as.matrix(sapply(sp, FUN = function(x) x[,"energy__"]))
            threshold <- 0.2
            if (nrow(E) > 1) {
              EBFMI <- get_num_upars(object) / apply(E, 2, var)
              n_e <- sum(EBFMI < threshold, na.rm = TRUE)
            }
            else n_e <- 0L
            if (n_e > 0)
              warning("There were ", n_e, 
                      " chains where the estimated Bayesian Fraction of Missing Information",
                      " was low. See\n", 
                      "https://mc-stan.org/misc/warnings.html#bfmi-low", call. = FALSE)
          }
          if (n_d > 0 || n_m > 0 || n_e > 0)

Presumably we want to use rstan::get_bfmi here @bgoodri @jonah

cabroncito · May 13, 2024, 8:44pm

Thank you @jsocolar I will move on assuming that there is a glitch in the “low BFMI” warning and check the BFMI directly.

Topic		Replies	Views
E-BFMI threshold or threshold(s)? General	2	896	July 9, 2018
Reparameterizing to avoid low E-BFMI warning Modeling fitting-issues	7	4421	October 2, 2022
How to address warnings about low ESS and low BFMI? Modeling	35	1023	February 14, 2024
Low E-BFMI: Reparameterization causing problems (low ESS, high R-hat) Modeling techniques , fitting-issues , specification	0	544	October 2, 2022
Bfmi-low problem with rstanarm after using parameters model_parameters rstanarm	0	527	March 18, 2022

Low BFMI warning with E-BFMI > 0.3

Related topics