Brms vs. Pymc3 - unexpected Brms results when using same priors

Interfaces brms
1

Hi

First time user of these forums. I’m currently working on developing three different models using multiple approaches and tools. The data sets are small enough (between 34 and 40 records of data) that most models I run, the solutions converge quickly.

All of my models are using multiple regression. I’m developing models / solutions using 1) brms 2) pystan 3) pymc3 4) rstan and 5) Constrained Non-Linear Optimization in Excel (GRG Method).

In a comparison today, I used the same priors in (strong informative) brms and pymc3. The results showed that brms solution did not change at all my priors (both the coefficient and its’ std. deviation, except for intercept that went negative) whereas the pymc3 solution showed greater movement (-22% to +32% range).

Since both are based on NUTS and use AD I expected the solutions to be some what similar, but the lack of change from the given priors in brms was a significant surprise to me.

I can share the necessary data and scripts, if you let me know what you need to see.

Sree

  • Operating System: Windows 10 -X64
  • brms Version: 2.4.0 (RStudio 1.1456 & R 3.5.1 via Anaconda 3)
  • pymc3 3.6 on Anaconda 3 using Python 3.6.7

P.S: for any one out there struggling to get RTools working with R and RStudio I can share my configuration if interested (if you do not have admin rights to change Windows System Environment (not user variables) there is no greater hell than that)

2

@paul.buerkner - would you be able to help with my above question / finding? After I posted my question here, I went back and reran the models in brms and pymc3 and the end result is the same. Brms estimates do not diverge from the strong priors I use.

I can share the exact data I’m working with if you need to replicate the problem via email since the data is sensitive.

Sree

3

I am on vacation until mid of next week. after that I will take a look at this issue.

4

Please enjoy your vacation.

Sree

5

This is likely because brms slightly reparameterizes the intercept coefficient for increased sampling effciency as documented in ?brmsformula. If you replace y ~ <predictors> by y ~ 0 + Intercept + <predictors> you can specify priors on the intercept directly, which then should yield the same results as the other approaches if everything else is correct.

6

@paul.buerkner Thanks a lot Paul. I should have looked at the docs more closely.

I’m working another dataset where I have to build a respondent level hierarchical model. I will apply what you mentioned in the original and the new dataset. I will let you know how that changes the estimation process and results.

Sree