Applying liklihood ratio test to models estimated using LBFGS

Hi Stan Community,

If I fit a model in Stan using L-BFGS (2 versions, with and without a parameter of interest), if the model has flat priors, is it valid to calculate a frequentist “p-value” for that parameter by using a likelihood ratio test comparing the 2 models?

I’m a stat modelling newbie sorry if this is a dumb question, but it seems to be that this should (probably be the case) but I can’t find the answer to this anywhere…


Here is something:
and here:

Thank you,

I’m trying to estimate about 20,000 models so typical Bayesian approach (e.g. MCMC) seems to be a little too slow. As a practical alternative I had hoped to build a fairly complex model in Stan, estimate the parameters using LBGFS, the try to approximate P-values using a liklihood ratio test (doing this 20,000 times). I know this isn’t a perfect approach, but I was hoping someone could tell me if its reasonable.

I think using Stan to fit LBFGS with flat priors would be essentially equivalent as estimating the model using conventional Maximum Liklihood Estimation? But is potentially a lot more convenient, due to Stan’s nice interface…

So the Priors, Posteriors, and P Values section of the Gelman paper doesn’t address this? He mentions: “Greenland and Poole’s proposal to either interpret one-sided P values as probability statements under uniform priors (an idea they trace back to Gossett) use one-sided P values as bounds on posterior probabilities (a result they trace back to Casella and Berger)”.

I think the simple answer is just ‘yes, this is fine’. There’s nothing inherently Bayesian about probability…


Thanks Charles. Thanks you increasechief for highlighting this section of the paper, its quite intimidating to a non-statistician.

1 Like