Loo moment match crashes on brms fits after combine_models

Interfaces brms
1

I am trying to run loo with moment_match = TRUE on models that were run separately and then combined using combine_models in brms. For context, each chain is run separately on different nodes of a cluster to take full advantage of within-chain parallelization using the cmdstanr backend. Regardless, I can replicate the issue by just running chains separately, combining them, and running loo with moment match on my local computer. I’ve provided a reproducible example below.

library(brms)

#run each chain separately within brms
brm_fit_c1 <- brm(
  count ~ zAge + zBase * Trt + (1|patient),
  data = epilepsy, 
  family = poisson(),
  chains = 1, 
  cores = 1,
  save_pars = save_pars(all=TRUE),
  seed = 022624 + 1
)

brm_fit_c2 <- brm(
  count ~ zAge + zBase * Trt + (1|patient),
  data = epilepsy, 
  family = poisson(),
  chains = 1, 
  cores = 1,
  save_pars = save_pars(all=TRUE),
  seed = 022624 + 2
)

brm_fit_c3 <- brm(
  count ~ zAge + zBase * Trt + (1|patient),
  data = epilepsy, 
  family = poisson(),
  chains = 1, 
  cores = 1,
  save_pars = save_pars(all=TRUE),
  seed = 022624 + 3
)

brm_fit_c4 <- brm(
  count ~ zAge + zBase * Trt + (1|patient),
  data = epilepsy, 
  family = poisson(),
  chains = 1, 
  cores = 1,
  save_pars = save_pars(all=TRUE),
  seed = 022624 + 4
)

#combine models 
brm_fit <- combine_models(
  brm_fit_c1,
  brm_fit_c2,
  brm_fit_c3,
  brm_fit_c4
)

#print summary
summary(brm_fit)

#run loo with moment match
brm_fit <- add_criterion(
  brm_fit,
  "loo",
  moment_match = TRUE
)

I’ve looked around the forum to find potential solutions. I’ve tried using cores = 1 and recompile = TRUE.

I also saw this post and tried this:

jloo <- loo(brm_fit)
loo_moment_match(brm_fit,jloo)

No matter what, I see the following error message:

Error in dim(pars) <- c(npars, ndraws) : 
  attempt to set an attribute on NULL
Error: Moment matching failed. Perhaps you did not set 'save_pars = save_pars(all = TRUE)' when fitting your model? If you are running moment matching on another machine than the one used to fit the model, you may need to set recompile = TRUE.

And just to cover my bases, the following code runs fine. I don’t have any moment_match issues when I fit the model all at once. It’s possible I’m misunderstanding something important about how moment match works, so may the behavior I’m describing is expected?

#fit model with 4 chains and 4 cores
brm_fit2 <- brm(
  count ~ zAge + zBase * Trt + (1|patient),
  data = epilepsy,
  family = poisson(),
  chains = 4,
  cores = 4,
  save_pars = save_pars(all=TRUE),
  seed = 022624 + 4
)

#print summary
summary(brm_fit2)

#loo with moment match
brm_fit2 <- add_criterion(
  brm_fit2,
  "loo",
  moment_match = TRUE
)

Any feedback/thoughts would be greatly appreciated! Thanks in advance!

  • Operating System: Mac OS 14.3.1
  • brms Version: 2.20.9
  • rstan Version: 2.32.5
  • stanHeaders Version: 2.32.5
2

Thanks for reporting this and providing a simple reproducible example

I found what is missing in the brms object created with combine_models(), and I created an issue

1 Like
3

Thanks for your help!

4

As a quick fix, you can run the following lines to copy the missing info

brm_fit$fit@sim$pars_oi_old <- brm_fit_c1$fit@sim$pars_oi_old
brm_fit$fit@sim$dims_oi_old <- brm_fit_c1$fit@sim$dims_oi_old
brm_fit$fit@sim$fnames_oi_old <- brm_fit_c1$fit@sim$fnames_oi_old
5

This should now be fixed in the github version of brms :-)

1 Like
6

Works perfectly. Thank you both!

1 Like