The se_diff
’s are not big and elpd_diff
’s are not big. It is possible that with more data the differences would stay similarly small. Low sample size makes it more difficult to learn the parameters well from the data and to see differences between the models. Weak priors on parameters may make the models with more parameters to perform worse than what would happen with priors that would provide consistent prior predictive distribution no matter how many parameters (although such priors exists, unfortunately there is not yet an easy way to add such priors for your models in brms). You could try with proper priors and then use priorsense
package to test prior and likelihood sensitivity.
There is no strict threshold, e.g. p_loo < 0.4*p
What is the range of values with the most of the posterior mass? Is the most of the mass on positive values or negative values? What is the probability that the quantity is positive/negative? What is the probability that the magnitude of the quantity is practically interesting? What is the interpretation of those values? Post a plot and tell about the quantity and I can help to interpret what we can read from the posterior plot
Ah, LOO-PIT can fail for discrete data, although it seems to failing in a way that shouldn’t happen.
As p_loo << n, these are also quite safe, especially the histogram one.