Can you confirm that this would be the correct method then?
> Main_EffectsModel=stan_glm(Accept_Reject~Discount+Floor,
+ family = binomial(link = "logit"),
+ data=sonadata_clean,
+ prior = Priors_MEmodel,
+ #prior_intercept = normal(),
+ #prior_PD = TRUE,
+ algorithm = c("sampling"),
+ mean_PPD = TRUE,
+ adapt_delta = 0.95,
+ #QR = FALSE,
+ #sparse = FALSE,
+ chains=3,iter=550,cores=3)
> lik=log_lik(Main_EffectsModel)
> loo::loo(lik, save_psis=TRUE,cores=16)
Computed from 825 by 633 log-likelihood matrix
Estimate SE
elpd_loo -231.6 16.3
p_loo 4.0 0.4
looic 463.3 32.6
------
Monte Carlo SE of elpd_loo is 0.1.
All Pareto k estimates are good (k < 0.5).
See help('pareto-k-diagnostic') for details.
Warning message:
Relative effective sample sizes ('r_eff' argument) not specified.
For models fit with MCMC, the reported PSIS effective sample sizes and
MCSE estimates will be over-optimistic.
The other difference is that brms uses loo.matrix by default while rstanarm uses loo.function. brms loo has the pointwise argument to switch to loo.function. When I activate that together with cores > 1, the loo computation does not terminate in reasonable time (few minutes) for a model that takes 2 seconds in pointwise evaluation when cores = 1. So there may be another problem with loo.function and cores on windows more generally. I will try to look into it later in more detail.
Nope, no errors, just disapointing benchmarks. After switching from an i5-6500 to a Ryzen 7 3700X I wanted to see how much more efficient my model run times were going to be; turns out the only benefit was the increased clockspeed of the newer CPU. Benchmarks stopped improving after setting “cores=3”.
Yeah that’s right. But you can leverage the extra cores by using certain features of the Stan language that allow for within-chain parallelization. Here’s a good tutorial from @bbbales2 on using one of those functions: