Comparing Bayesian and Frequentist Models

I’ve seen posts and read about WAIC and LOO metrics for comparing Bayesian models. What if I’d like to compare a brmsfit object to a model obtained with, say, survreg? If I call WAIC and pass objects referring to those two models, it seems to ignore the latter.

  • Operating System: Windows 7 (64-bit)
  • brms Version: 2.4.0

I am surprised that it does not throw an error message. In any event, don’t use WAIC; use LOO. But neither of them are going to work with an object produced by survreg because survreg does not output draws from a posterior distribution.

Yes, I wasn’t expecting that to work (just wanted to see what I’d get!). But I guess the main question is: is there a way to compare different types of models (i.e., a Bayesian one vs. a frequentist one)?

Also, why did you suggest the following?

In any event, don’t use WAIC; use LOO.

At best, WAIC and LOOIC yield the same estimates but LOOIC works better with small amounts of data and has diagnostics that tell you when its assumptions are violated.

In my opinion, there is no criterion on which you could compare a Bayesian and a frequentist model that would be acceptable to both Bayesians and frequentists. Others might disagree, but it certainly wouldn’t be LOOIC because that involves evaluating the likelihood over the posterior distribution.