Maximum Likelihood estimation for Random Effect model for Meta Analysis

Modeling
,
21

There is but it’s quite old, I’m working on a version right now which I think we should have out by next release

3 Likes
22

Awesome!

23

I am all for a fully Bayesian inference @andrewgelman, which is what we do in the paper (it is the UCT meta analysis with Dean and Chris), but the Econ audience is always more comfortable if provided also an ML estimate, to compare it to the Bayesian result. I am now wondering what type of ML estimation procedure packages commonly used for meta analyses such as metafor do, since they estimate Random Effect hierarchical models…

Which is why I wanted to do it through Stan to be sure about what model and procedure I am using, but I guess it is more subtle than what I thought, thank you very much for all the feedbacks.

24

Hi @jonah, just to summarize as not to make confusion:

  1. If I use the rstan package without stating priors on hyper-parameters, then the MAP is not the MLE because Stan uses transformation on constrained parameters
  2. If I use the cmdstanr package, I can turn off the Jacobian adjustment (it is already off by default), thus obtaining the MLE
  3. The MLE that I obtain with this procedure is that of case (a), as explained by @andrewgelman

I did some experimenting and I obtain the same numerical result up to the third decimal with 1. and 2., is it just a coincidence?

25

I continue to emphasize that if you try to obtain the joint maximum likelihood estimate for a hierarchical model, you will fail because the mode does not exist–it blows up at group level variance approaching 0. If your computer program converges to something, this is a local mode, not the global mode. It could be that for computational reasons you can get to the same local mode using different algorithms. It is incorrect to call this a maximum likelihood estimate or posterior mode, as it is not the maximum of the likelihood and it is not the posterior mode.

1 Like
26

I agree with this point of emphasis from Andrew here.

Oh, I think these are actually doing the same thing (although likely with different versions of Stan under the hood because rstan is a few versions behind). If I recall correctly, rstan doesn’t use the jacobian adjustment and it doesn’t give you an option to choose (because like @Bob_Carpenter said, MAP was only added recently). With cmdstanr’s ability to use the latest Stan version, you have the choice, so you could compare the results setting jacobian to TRUE and FALSE and see a difference if there are constrained parameters. Sorry for the confusion!