Stan_glm

Modeling
1

I’ve been running stan_glm for over 48 hrs and it hasn’t budged.

mod_stan <- stan_glm(stanForm, data=TrFull,
+                      family=gaussian, prior=lasso(), iter=1000)

SAMPLING FOR MODEL 'continuous' NOW (CHAIN 1).
Chain 1:
Chain 1: Gradient evaluation took 2.86364 seconds
Chain 1: 1000 transitions using 10 leapfrog steps per transition would take 28636.4 seconds.
Chain 1: Adjust your expectations accordingly!
Chain 1:
Chain 1:
Chain 1: Iteration:   1 / 1000 [  0%]  (Warmup)

The models has 23 predictors and the training set has about 350,000 rows. Any suggestions?

2

Does a random subset of your data run? Like a few hundred rows?

3

I would have used stan_lm with that many observations in a linear model. Its prior shrinks the coefficients heavily as well.

1 Like
4

What’s the difference between a gaussian stan_glm and stan_lm? Is it just the priors? Is there anything online I can peruse? Do they both use Hamiltonian MC?

5

There is a vignette for it (and most other things)
https://cran.r-project.org/web/packages/rstanarm/vignettes/lm.html
Everything in rstanarm uses Stan’s default MCMC algorithm (which is NUTS rather than HMC with static integration time), unless you specify the algorithm argument differently.

6

I agree with Ben on stan_lm.

Even if using stan_glm or less data, don’t use prior=lasso(). See https://andrewgelman.com/2017/11/02/king-must-die/ and https://betanalpha.github.io/assets/case_studies/bayes_sparse_regression.html

With this much data and so few predictors you are fine with normal prior. For faster inference, it’s likely that you get practically the same accuracy with algorithm=“optimizing”. There will be soon a function implementing diagnostics for this.

7

I was able to fit a model using

mod_stan <- stan_lm(...)

but have not had luck using shinystan.

launch_shinystan(mod_stan)

does not launch anything in the browser of my Ubuntu 16.04 in firefox using shiny 1.1.0.

8

launch_shinystan is going to take a long time since it has to do posterior predictions for all observations. If you look at the help file, there is some way to make it skip that but then it is less useful.