Multi-membership random effect has low ESS

Modeling
,
1

I’m having some trouble fitting a multi-membership model in brms.

The data:
The response variable consists of sample similarity values derived from a pairwise distance matrix (there are ~37000 pairwise comparisons of 462 samples). Similarity values are bounded by 0 and 1 but don’t include 0 and 1- hence I think a beta regression seems appropriate.

Predictor variables are ageclass_combination (a factor), sex_similarity (binary, 0=same, 1= different), and temporal distance (numerical, but scaled between 0-1).

I’ve also included two multi-membership random effects to control for the structure of the data. The first (1|mm(IDA,IDB)) controls for the fact that each sample occurs in multiple pairwise comparisons and so comparisons involving the same sample are not independent. The second (1|mm(BirdIDA,BirdIDB)) controls for the fact that some individuals (~50%) have multiple samples, and so comparisons involving the same individual (birds in this study) are also not independent.

My model structure is as follows:

model1<-brm(sample_similarity~ 1 + ageclass_combination +
                sex_similarity + Temporal_distance +
                + (1|mm(IDA,IDB)) + (1|mm(BirdIDA, BirdIDB)),  
              data = data.dyad, 
              family= "Beta", prior=priors,
              warmup = 1000, iter = 3000, 
              cores = 4, chains = 4,
              init=0)

At the moment I have imposed a regularising normal(0,1) prior on b. The intercept and sd have a student_t(3,0,2.5) prior.

When I run the model I get very low bulk/tail ESS values for the BirdID group level effect and rhat is 1.06. Trace plots show chains haven’t mixed well for this parameter. Also when I run pp_check() the posterior draws don’t seem to align very well with the data.

image
image2245×1366 307 KB

image
image2113×1918 176 KB

I would really appreciate any advice on

  1. Whether I’ve specified the model correctly given the data
  2. How I might proceed from here in order to improve convergence/model fit (I’m new to running bayesian analyses). I’m not sure I understand how to set more informative priors on group level effects in a beta regression (and if this is the problem in the first place).
1 Like
2

Hi,
there is a host of things you can try to understand what the problem is. A typical problem would be that the data do not really let you learn about the parameters - either there is too few datapoints or the structure of the data does not let you learn something (e.g. collinearity of predictors) and no amount of data would help.

A good first step is to try to reduce the model: what is the biggest model that runs OK?

Some broader discussion (aimed at users of Stan directly, so only partially applicable to brms) is at Runtime warnings and convergence problems

Hope that helps at least a little.

1 Like