Multilevel Logit with Few Observations per Individual

[martinmodrak]

Sorry for once again missing your question. A few things to note:

• You seem to require large adapt_delta and max_treedepth - presumably to avoid divergencies. You then - if I understand you correctly - get rid of divergencies but get BFMI warnings. This is likely just a different manifestation of the same problem - for diagnostic purposes, it is usually better to run with default settings as it shows you more of the problems.

@alexkellogg

Is it appropriate to use multilevel models to recover the latent λ\lambda population distribution?

I don’t think this is fundamentally problematic, but can definitely be a problem in practice for some specific settings.

@alexkellogg

Empirically, the posterior on ¯λ\bar{\lambda} is quite good even when D=1D=1 – so that I only observe one choice per individual. Is this a lucky coincidence, or can I actually hope to recover the population distribution even with 1 choice per individual (where I can’t hope to recover the individual λi\lambda_i )?

Once again - this is not something that is IMHO problematic in principle but might be a problem in practice.

@alexkellogg

I am running into a “BFMI Low” warning when I increase the sample size from N=100 to N=1000 (particularly at D = 1, 2, and 3). The E-FMI is around 0.15 according to “check_energy()”. Why would that be?

As I said above, I believe the actual problem is not BFMI, so I would not fixate too much on this. (I admit I do not understand BFMI much, so I don’t want to speculate)

@alexkellogg

I’m also not entirely sure how to compute the pairs() plot, which I think are a suggested diagnostic for the BFMI warning, since there are so many λi\lambda_i in the model.

I would just look at the pairs plot for a bunch of different subsets of the variables. Note also additional debugging strategies at Divergent transitions - a primer. I would start by making sure you can fit a simpler version (e.g. non-hierarchical) of the model without ramping up adapt_delta or max_treedepath

Additionally:
The hard lower and upper bounds are suspicious - why do you believe such hard bounds are sensible a priori?

Best of luck with the model!