Hello,
I am trying to build a model using meanfield algoritm, but get an error Error in draws[[1]] : subscript out of bounds. In examples below, I am using brms 2.16.0, but the same problem holds for 2.16.1, 2.16.3 versions too.
My code is represented below:
rm(list = ls())
library(tidyverse)
library(cmdstanr)
library(rstan)
library(brms)
tryCatch({cmdstanr::set_cmdstan_path('/home/cmdstanr/cmdstan-2.27.0')})
# ------------------------------------------------------------------------------
# Generate data
# ------------------------------------------------------------------------------
N <- 1000
nchains = 3
ncores = 3
nthreads = 8
data0 <-
tibble(
x = rnorm(N, 0, 1), # x = F1
x1 = rnorm(N, 1*x, 1),
x2 = rnorm(N, 0.5*x, 1),
x3 = rnorm(N, 1*x, 1),
F1 = as.numeric(NA)
)
bf0 <- bf(x1 ~ 0 + mi(F1)) +
bf(x2 ~ 0 + mi(F1)) +
bf(x3 ~ 0 + mi(F1)) +
bf(F1 | mi() ~ 1) +
set_rescor(rescor = FALSE)
prior0 <- prior(constant(1), class = "b", resp = "x1") +
prior(constant(1), class = "sigma", resp = "x1") +
prior(normal(0, 10), class = "b", resp = "x2") +
prior(constant(1), class = "sigma", resp = "x2") +
prior(normal(0, 10), class = "b", resp = "x3") +
prior(constant(1), class = "sigma", resp = "x3") +
prior(normal(0, 10), class = "Intercept", resp = "F1") +
prior(cauchy(0, 1), class = "sigma", resp = "F1")
# ------------------------------------------------------------------------------
# Variatonal inference without threads
# ------------------------------------------------------------------------------
m1 <-
brm(
formula = bf0,
prior = prior0,
data = data0,
iter = 500,
backend = "cmdstanr",
algorithm = 'meanfield',
control = list(tol_rel_obj = 0.00000001),
refresh = 5
)
m1
Output I get:
Compiling Stan program...
Start sampling
------------------------------------------------------------
EXPERIMENTAL ALGORITHM:
This procedure has not been thoroughly tested and may be unstable
or buggy. The interface is subject to change.
------------------------------------------------------------
Gradient evaluation took 0.000455 seconds
1000 transitions using 10 leapfrog steps per transition would take 4.55 seconds.
Adjust your expectations accordingly!
Begin eta adaptation.
Iteration: 1 / 250 [ 0%] (Adaptation)
Iteration: 50 / 250 [ 20%] (Adaptation)
Iteration: 100 / 250 [ 40%] (Adaptation)
Iteration: 150 / 250 [ 60%] (Adaptation)
Iteration: 200 / 250 [ 80%] (Adaptation)
Success! Found best value [eta = 1] earlier than expected.
Begin stochastic gradient ascent.
iter ELBO delta_ELBO_mean delta_ELBO_med notes
100 -4985.239 1.000 1.000
200 -4924.670 0.506 1.000
300 -4915.218 0.007 0.012
400 -4909.433 0.002 0.002
500 -4910.086 0.001 0.001
Informational Message: The maximum number of iterations is reached! The algorithm may not have converged.
This variational approximation is not guaranteed to be meaningful.
Drawing a sample of size 1000 from the approximate posterior...
COMPLETED.
Finished in 0.8 seconds.
Error in draws[[1]] : subscript out of bounds
Run `rlang::last_error()` to see where the error occurred.
rlang::last_error() returns:
> rlang::last_error()
<error/rlang_error>
Error in draws[[1]] : subscript out of bounds
Backtrace:
1. brms::brm(...)
2. brms::do_call(fit_model, fit_args)
3. brms:::eval2(call, envir = args, enclos = envir)
4. [ base::eval(...) ] with 1 more call
6. brms:::.fun(...)
7. brms:::.fit_model(model, ...)
8. cmdstanr::read_cmdstan_csv(...)
9. base::do.call(...)
Run `rlang::last_trace()` to see the full context.
rlang::last_trace() returns:
> rlang::last_trace()
<error/rlang_error>
Error in draws[[1]] : subscript out of bounds
Backtrace:
x
1. \-brms::brm(...)
2. \-brms::do_call(fit_model, fit_args)
3. \-brms:::eval2(call, envir = args, enclos = envir)
4. \-base::eval(expr, envir, ...)
5. \-base::eval(expr, envir, ...)
6. \-brms:::.fun(...)
7. \-brms:::.fit_model(model, ...)
8. \-cmdstanr::read_cmdstan_csv(...)
9. \-base::do.call(...)
> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.3 LTS
Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.8.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8
[6] LC_MESSAGES=C LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] forcats_0.5.1 stringr_1.4.0 purrr_0.3.4 readr_2.1.1 tidyr_1.1.3 tibble_3.1.6 tidyverse_1.3.1
[8] qs_0.24.1 brms_2.16.0 Rcpp_1.0.8 rstan_2.26.6 ggplot2_3.3.5 StanHeaders_2.26.6 cmdstanr_0.4.0
[15] data.table_1.14.2 dplyr_1.0.7
loaded via a namespace (and not attached):
[1] minqa_1.2.4 colorspace_2.0-2 ellipsis_0.3.2 ggridges_0.5.3 rsconnect_0.8.25 markdown_1.1 fs_1.5.2
[8] base64enc_0.1-3 rstudioapi_0.13 farver_2.1.0 DT_0.18 lubridate_1.8.0 fansi_1.0.2 mvtnorm_1.1-3
[15] xml2_1.3.3 bridgesampling_1.1-2 codetools_0.2-18 splines_4.1.0 knitr_1.37 shinythemes_1.2.0 bayesplot_1.8.1
[22] projpred_2.0.2 jsonlite_1.7.3 nloptr_1.2.2.3 broom_0.7.11 dbplyr_2.1.1 shiny_1.7.1 httr_1.4.2
[29] compiler_4.1.0 backports_1.4.1 assertthat_0.2.1 Matrix_1.3-3 fastmap_1.1.0 cli_3.1.1 later_1.3.0
[36] htmltools_0.5.2 prettyunits_1.1.1 tools_4.1.0 igraph_1.2.11 coda_0.19-4 gtable_0.3.0 glue_1.6.1
[43] reshape2_1.4.4 posterior_1.2.0 V8_4.0.0 cellranger_1.1.0 vctrs_0.3.8 nlme_3.1-152 crosstalk_1.2.0
[50] tensorA_0.36.2 xfun_0.29 ps_1.6.0 rvest_1.0.2 lme4_1.1-27.1 mime_0.12 miniUI_0.1.1.1
[57] lifecycle_1.0.1 gtools_3.9.2 MASS_7.3-54 zoo_1.8-9 scales_1.1.1 colourpicker_1.1.1 hms_1.1.1
[64] promises_1.2.0.1 Brobdingnag_1.2-6 parallel_4.1.0 inline_0.3.19 shinystan_2.5.0 gamm4_0.2-6 curl_4.3.2
[71] gridExtra_2.3 loo_2.4.1 stringi_1.7.6 dygraphs_1.1.1.6 checkmate_2.0.0 boot_1.3-28 pkgbuild_1.3.1
[78] rlang_0.4.12 pkgconfig_2.0.3 matrixStats_0.61.0 distributional_0.3.0 lattice_0.20-44 rstantools_2.1.1 htmlwidgets_1.5.4
[85] processx_3.5.2 tidyselect_1.1.1 plyr_1.8.6 magrittr_2.0.1 R6_2.5.1 generics_0.1.1 DBI_1.1.2
[92] haven_2.4.3 pillar_1.6.4 withr_2.4.3 mgcv_1.8-35 xts_0.12.1 abind_1.4-5 modelr_0.1.8
[99] crayon_1.4.2 utf8_1.2.2 RApiSerialize_0.1.0 tzdb_0.2.0 readxl_1.3.1 grid_4.1.0 callr_3.7.0
[106] threejs_0.3.3 reprex_2.0.1 digest_0.6.29 xtable_1.8-4 httpuv_1.6.5 RcppParallel_5.1.5 stats4_4.1.0
[113] munsell_0.5.0 stringfish_0.15.5 shinyjs_2.1.0
I saw that there is closed issue in github with the same problem as mine (Error using meanfield and fullrank algorithms when using cmdstanr backend · Issue #1189 · paul-buerkner/brms · GitHub), but I can’t put my head around why this problem is still relevant in my case. P.s. I tried run code provided by issue creator, got the same output with error.