Differences in predictions between Ubuntu 18.04 and 20.04 with brms for model with splines

G’day Stan/brms community -

I’m running into a problem with brms that I can’t seem to solve and would be very grateful for any pointers; I ran a model on a remote (AWS) machine using a ubuntu 18.04 docker image, then ran model checks on both the remote machine and my local machine (from the same brmsfit object), both running brms 2.16.1 but using ubuntu 20.04 (as well as 21.04) locally.

The predictions - verified using a couple of different ubuntu 18.04 dockers - work fine on the 18.04 environments (1st plot below), but return lots of NAs and weird predictions in 20.04 (using the same model fit object downloaded from the remote machine - 2nd plot; only non-NA draws plotted). Tried a number of rstan versions (2.21 - 2.28) locally on 20.04, all give the same predictions - all seem way off. Also tried with different brms etc, and verified that the mgcv backend that (I think) is used for the t2() smooth formulations in the model is the same across systems.


I suspect that it to do with some linked library, but have run out of ideas on where to look.

The model itself is this:


mlf <- bf(mean_L | trunc(lb = 0) + weights(rel_catch)  ~ 
            primary_method + 
            t2(mchla,bs = c('tp'))+ 
            t2(start_seabed_depth,SeasonWeek,fyear,bs = c('tp','cc','re'))+ 
            t2(start_latitude,start_longitude,SeasonWeek,fyear,d=c(2,1,1),bs = c('tp','cc','re')))


run via

priors <- set_prior("normal(0,10)", class='b')+
  set_prior("normal(0,100)", class='sds')


t2 <- brm(mlf,
          data=lf_by_tow,
          cores=32,
          threads=8,
          chains=4,
          thin=3,
          warmup=250, 
          iter = 1000,
          inits = "0",
          backend = 'cmdstanr',  
          prior = priors,
          refresh=10)

Curiously, I can’t reproduce the behavior with a simpler model - e.g., the model from the s() documentation on brms runs fine; and produces OK pp_check results. Similarly, tweaking that model to involve t2() seems to work fine locally:

dat <- mgcv::gamSim(1, n = 200, scale = 2) %>%  filter(y>0)
fit1 <- brm(y|trunc(lb=0) + weights(x1*10) ~ t2(x0,x1,x2,x3,d=c(2,1,1),bs=c('cr','cc','re')), 
            data = dat, chains = 2)

However, with the original model object, colleagues have run into the same issue with windows and 20.04 with less recent R installs (R 3.6.3 vs 4.1 locally on my machine and in dockers.).

BRMS object is here; any pointers would be greatly appreciated!

thanks!

1 Like

Do the same problems show up without docker? If so, would you mind filing a bug report against BRMS?

The place to report bugs is on the GitHub issue tracker for brms:

That’ll make sure it’s on the radar of the brms devs who might not see it here.