Problems with default prior for family = "binomial"


#1

I use Windows 7, RStudio, R version 3.5.1, ‘brms’ package (version 2.3.4) from github.
This tutorial gave me working template (fixing labels after hypothesis() is a minor issue).

require(tidyverse) 
require(brms) 
d <- data.frame(s = 9, k = 10)

mOK <- brm(s | trials(k) ~ 0 + intercept, data=d,
           prior = set_prior("beta(1, 1)", class = "b", lb = 0, ub = 1),
           family = binomial(link="identity"), sample_prior = TRUE, cores = 3,
           seed = 20180709 )

(hOK <- hypothesis(mOK, "intercept = 0.5"))
##              Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Star
## 1 (intercept)-(0.5) = 0     0.33      0.11     0.09     0.48       0.14    *

plot(hOK, plot = F, theme = theme_get())[[1]] +
  scale_x_continuous(breaks = seq(-.5, .5, by = .25), 
                     labels = seq(-.5, .5, by = .25)+.5)

Now I tried to use default prior and got two problems:

mNoBF <- brm(s | trials(k) ~ 0 + intercept, data=d, 
  family = binomial(link="identity"), sample_prior = TRUE, iter = 4e4, cores = 3,
  seed = 20180709, control=list(max_treedepth=10, adapt_delta=0.999))

It leads to “Rejecting initial value” warning (hidden in RStudio’s Viewer), but at least we get a warning about “2988 divergent transitions after warmup” to alert us. Increasing adapt_delta does not help even for this embarrassingly simple model. My real binomial (2-way mixed) model barely gives me ESS=918 with 4e5 iterations over 4 cores and tons of “divergent transitions” using default prior.
Question #1: Could the default binomial prior be chosen better?

(hNoBF  <- hypothesis(mNoBF,  "intercept = 0.5"))
#              Hypothesis Estimate Est.Error CI.Lower CI.Upper Evid.Ratio Star
# 1 (intercept)-(0.5) = 0     0.33       0.1     0.09     0.48         NA    *

plot(hNoBF)

Question #2: Was “sample_prior = TRUE” simply ignored in mNoBF <- brm() without warning? Why not sample default prior, just like it is done for user-given prior, especially if hypothesis() needs it to compute Evid.Ratio?

hNoBF gives the same results as hOK, so I guess Stan (as usual) is too “trigger happy” to fire these divergent transitions warnings on us…


#2

Answer 1: Default priors are not chosen with Bayes factors in mind. If you want to reasonably use Bayes factors, please specify the prior yourself.

Answer 2: The default prior on the intercept if you use ~ 0 + intercept is improper flat (i.e. “no prior” if you want to say it like this). So this prior is not actually sampled from and does not lead to the divergent transitions. You get the latter because you use a non-bounded parameter to estimate something between 0 and 1. Remember that you didn’t specify boundaries for the intercept in mNoBF but still used the identity link.