Poor convergence using truncated count model vs. equivalent hurdle model

Interfaces brms
1

I’m using brms to fit poisson models to count data that are truncated at some lower interval, but having convergence problems when I specify a formula using the trunc() function.

For example, if I generate a data set in which the zeros are snipped out:

obs <- rpois(300, lambda = 1.5)
dat2 <- data.frame(obs)
dat2_filt = data.frame(obs = dat2[dat2$obs > 0, ])

and then fit a truncated poisson model as follows:

formula.truncpois = brmsformula(obs | trunc(lb=1) ~ 1, family = poisson())
fit_truncpois <- brm(formula.truncpois, data = dat2_filt)

I am unable to get a fit that correctly estimates the value of lambda that I used to generate the data. In this case, I can work around the issue by fitting a hurdle model and fixing the hurdle probability to zero:

formula.hpois = brmsformula(obs ~ 1, hu = 0, family = hurdle_poisson())
fit_hpois <- brm(formula.hpois, data = dat2_filt)

And this works fine. I get a good estimate from this model.

I may be confused, but I understand these two models to be functionally equivalent to one another. I examined the generated stan code, and though I’m still learning how to read its syntax, was able to see that brms has generated different code for each situation. I’ve posted the full code for both models in this gist for completeness.

The workaround has been functional for me so far, but I’d like to eventually adjust the truncation bound to some value other than 1.

Am I using the trunc() function improperly, or in the wrong context? Thank you for your help.

  • Operating System: macOS 10.12.6
  • brms Version: 2.7.2 and 2.7.0
2

It’s possible that due to the differing ways the model is written down, convergence varies. Can you try it again with the github version of brms. I just made a fix that could perhaps help with your problem.

1 Like
3

I tried out the github version today and your changes have fixed the problem! I can now specify this model using trunc and can freely adjust the lower bound to truncate values other than zero. Thank you!

Out of curiosity, I peeked at your changes in the git repo. It looks like you have shifted the lower bound specified in brms (which is inclusive) before passing it as input to STAN’s parameterization of the complementary CDF (which is apparently exclusive). Without subtracting one from the argument, I imagine the probability mass values in the (unlogged) likelihood function were being rescaled by a denominator that was too small. I’m at the limit of my understanding here, so I’m not sure if that’s a fair characterization, or how that could cause the convergence problem — but this was interesting for me to examine.

I’m grateful for your work to fix this. Thank you.

4

Your characterization of my fix is correct. I am actually surprised it also solves your convergence problems.