# Reference on benefits of rstanarm GAM models over frequentist alternatives?

**URL:** <https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114>\
**Category:** rstanarm\
**Tags:** performance\
**Created:** [April 6, 2020, 9:13am UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114 "2020-04-06T09:13:55Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![benskov](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/benskov/32/7580_2.png) [@benskov](https://discourse.mc-stan.org/u/benskov)\
**Post date:** [April 6, 2020, 9:13am UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114/1 "2020-04-06T09:13:55Z")

</div>

Hi everybody,

I would like to use `rstanarm::stan_gamm4` for a study we’re preparing, and to substantiate my choice of Bayesian parameter estimation (unfortunately, this is still needed) I’ve come across some these claims and was wondering if there were any references to back them:

- “The documentation of lme4 and gamm4 has various warnings that acknowledge that the estimated standard errors, confidence intervals, etc. are not entirely correct, even from a frequentist perspective.  
A frequentist point estimate would also completely miss the second mode in the last example with stan\_nlmer. Thus, there is considerable reason to prefer the rstanarm variants of these functions for regression modeling. The only disadvantage is the execution time required to produce an answer that properly captures the uncertainty in the estimates of complicated models such as these.” (Conclusion in [link](https://mc-stan.org/rstanarm/articles/glmer.html#relationship-to-gamm4))
- “Estimating these models via MCMC avoids the optimization issues that often crop up with GAMMs and provides better estimates for the uncertainty in the parameter estimates.” (last sentence of first paragraph of the Details in the [documentation](https://www.rdocumentation.org/packages/rstanarm/versions/2.19.3/topics/stan_gamm4))

If anyone has such references, examples or similar, I’d be keen to know of their existence.

Cheers,  
Ben

---

<div class="post-metadata">

**Author:** ![emiruz](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/emiruz/32/5518_2.png) [@emiruz](https://discourse.mc-stan.org/u/emiruz)\
**Post date:** [April 6, 2020, 11:29am UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114/2 "2020-04-06T11:29:54Z")

</div>

Why not do both? If you’re going to be cross validating your model then a good argument might be to show that the Bayesian version fits the data better in k-fold cross validation or whatever.

---

<div class="post-metadata">

**Author:** ![benskov](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/benskov/32/7580_2.png) [@benskov](https://discourse.mc-stan.org/u/benskov)\
**Post date:** [April 20, 2020, 9:51am UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114/3 "2020-04-20T09:51:34Z")

</div>

The point of the model is not predicting but causal inference on the effect estimates, so we don’t plan on using cross validation; only if PSIS-LOO fails.

The point on incorrect errors directly implies that using the implementation in `rstanarm` is desirable for the purpose of our analysis, so it would be nice to have a proper reference on this point.

Albeit interesting, the point on convergence problems is more technical and would only become relevant if the model one would fit turned out to fail.

---

<div class="post-metadata">

**Author:** ![maxbiostat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/maxbiostat/32/1326_2.png) [@maxbiostat](https://discourse.mc-stan.org/u/maxbiostat)\
**Post date:** [April 20, 2020, 12:52pm UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114/4 "2020-04-20T12:52:05Z")

</div>

> [@benskov](#):
>
> The point on incorrect errors directly implies that using the implementation in `rstanarm` is desirable for the purpose of our analysis, so it would be nice to have a proper reference on this point.

You can always run [simulation-based calibration](https://arxiv.org/abs/1804.06788) and show that the model-fitting strategy you’re using is “consistent”.

---

<div class="post-metadata">

**Author:** ![benskov](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/benskov/32/7580_2.png) [@benskov](https://discourse.mc-stan.org/u/benskov)\
**Post date:** [April 20, 2020, 12:56pm UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114/5 "2020-04-20T12:56:06Z")

</div>

Thanks for the pointer, @maxbiostat! Will look into that.

---

<div class="post-metadata">

**Author:** ![maxbiostat](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/maxbiostat/32/1326_2.png) [@maxbiostat](https://discourse.mc-stan.org/u/maxbiostat)\
**Post date:** [April 20, 2020, 12:58pm UTC](https://discourse.mc-stan.org/t/reference-on-benefits-of-rstanarm-gam-models-over-frequentist-alternatives/14114/6 "2020-04-20T12:58:22Z")

</div>

If you decide to go down that road, maybe @bgoodri or @jonah can give some insight into how to do SBC with **rstanarm**.