Hello, I a fitting a hierarchical model in brms for an ecological meta-analysis. I am interested in magnitude of the effect size so for the response in the model I am using the absolute values of the effects sizes as opposed to both the positive and negative effects. This makes perfect biological sense for the study, which I won’t go into here. See the two plots below for a visual representation of this:
Before absolute value
After absolute value:
As you can see from the second plot, the distribution of the data I am fitting is mostly near zero but will never be below zero with a tail extending out to ~14. As such, I have been playing around with a couple families (lognormal, beta, and normal (which is very clearly wrong, but there for comparison)) to see which distribution would be best for my data. It also seems like halfnormal or negative exponential distributions may be appropriate, neither of which are native to brms but I would be interested in trying to use with leave one out or kfold model comparison.
Because the beta distribution inherent to brms works on probability it’s bounded between 0 and 1. I know from a couple of sources (Ben Bolker’s book in particular) that the beta distribution is extensible beyond 1, and I would like to make a custom family that does so that more appropriately matches the distribution of my data. I’ve looked at this resource https://cran.r-project.org/web/packages/brms/vignettes/brms_customfamilies.html but unfortunately don’t have enough experience with this stuff to move from the provided example to a beta distribution.
Currently, as a work around to be able to compare models using leave one out (
loo() in brms) I am fitting my intercept only model where the response is divided by 100. This seems obviously wrong, but I am not sure how wrong. Here is a quick bite of that code without the other model specifications
brm( effect.size/100 ~ 1 + (1|Paper.Name/Trait/Experiment))
Using this method, loo() suggests that the beta distribution is the better preforming model:
Model comparisons: elpd_diff se_diff my_model_int_beta 0.0 0.0 my_model_int_lognorm -435.9 4.1
Also, strangely, when I run the model with a lognormal distribution I end up with a mean posterior distribution less than zero, which seems a bit strange given the bounds of the family (and that it appears there is no link function), but like I have indicated earlier this all fairly new to me.
Thanks in advance for your help.