I’m trying to fit a hierarchical model to some price data in Stan. The model is basically a random effects model with partially pooled dummies for every product type (UPC) and county (stfips) in a particular area of the United States. The dependent variable is log of unit price for a product. There are on the order of ~140,000 observations of individual transactions aggregated over some time period.
Unfortunately I can’t attach the data or be much more specific due to confidentiality concerns. But the Stan program I’m attempting to run is attached here: p_model_8.stan (2.4 KB)
This model had successfully fit before on a subset of the data that I am attempting to use now. However, once I expanded the data to include more and slightly different types of products, I began to have issues.
Specifically, I encounter two problems that I can’t seem to explain:
- The model takes 1-2 days to fit.
- More significantly, the fit always has several of the Upc_Epsilon terms failing to converge between chains, with Rhat values >1.1. The Upc_Sigma term also consistently has high Rhat values. These are generally on the order of between 1.2 and 1.7.
The second problem in particular flummoxes me, since the Upc_Epsilon terms should just be normally distributed noise that doesn’t depend on the data. The first problem makes it hard to run multiple models and debug problems.
Things I have attempted so far to diagnose this problem:
- Increased maximum treedepth as is discussed here.
- Removed the Upc_Sigma term from the model. Obviously this removes the Upc_Sigma’s convergence issues, but the Upc_Epsilon terms still have high Rhat values.
- Changed priors on the sigma terms from Cauchy priors to student-t priors with 3 degrees of freedom, and, when that failed, changed the priors to gamma(2, 0.5) priors in line with a recommendation I read in this Psychometrika paper.
None of these attempts have resolved the issue so far. Being a relatively new user of Stan, I’m not quite sure how to proceed from here. Any suggestions for fixing this are greatly appreciated.
Additionally, this is my first post on this forum. I’ve read the forum etiquette guidelines and searched for similar posts before posting this question, but if I’ve missed anything please let me know and I will try to correct it.