If I have k-fold (k=10) cv that I am running with the same model object for every fold, but different training data. Should I be able to extract an inv_metric diagonal mass matrix from the first fold and then apply it to all the subsequent runs?
options(cmdstanr_write_stan_file_dir = here::here('script/pipelines/Model/stan'))
brms_object <- brm(model,
prior = model_priors,
chains = 0,
backend = "cmdstanr",
data = full_data
)
fit_obj <- update(brms_object,
warmup = 1000,
iter = 1000,
chains = 1,
save_warmup =TRUE,
seed = 12345,
init = runif(1, 0, 1e-2),
sig_figs = 18,
newdata = data_fold_1)
brms_inv_mat <- (fit_obj[["fit"]] |>
rstan::get_adaptation_info() |>
(\(x) gsub('\\# ','',x))() |>
strsplit('\n'))[[1]][3] |>
(\(x) sprintf('c(%s)',x))() |>
(\(x) parse(text = x))() |>
eval() |>
as.vector()
update(brms_object,
iter = 2000,
thin = 2,
cores = 4,
seed = 12345,
init = stats::runif(1, 0, 1e-2),
inv_metric = brms_inv_mat,
newdata = data_fold_2
)
when I try to do it with the cmdstan engine i get this error
The desired updates require recompiling the model
Model executable is up to date!
Start sampling
Running MCMC with 4 parallel chains...
Warning: Chain 1 finished unexpectedly!
Warning: Chain 2 finished unexpectedly!
Warning: Chain 3 finished unexpectedly!
Warning: Chain 4 finished unexpectedly!
Warning: Use read_cmdstan_csv() to read the results of the failed chains.
Error: Fitting failed. Unable to retrieve the metadata.
In addition: Warning messages:
1: All chains finished unexpectedly! Use the $output(chain_id) method for more information.
2: No chains finished successfully. Unable to retrieve the fit.
When i generate the inv_metric for each fold I can see they are all different lengths. I tried to read the documentation about the mass matrix calculation, but do not quite understand why it would differ in length when the model is constant.
> purrr::map_dbl(
result2$result, ~ (.x$fit |>
rstan::get_adaptation_info() |>
(\(x) gsub('\\# ','',x))() |>
strsplit('\n'))[[1]][3] |>
(\(x) sprintf('c(%s)',x))() |>
(\(x) parse(text = x))() |>
eval() |>
as.vector() |> length())
[1] 992 995 993
Running each pair of fold/inv_metric does work and does reduce the run times as expected (400s==>200s)