Hi all,
I am curious how I can estimate the time for running if I get the following message from rstan: “Gradient evaluation took 0.015 seconds
1000 transitions using 10 leapfrog steps per transition would take 150 seconds.
Adjust your expectations accordingly!”
With this if I run 4000 iteration with three chains (parallel, each chain per CPU) then how much time do I need?
Thank you for your answer!
Tran.
I don’t think that gives you enough information to find the required time, as it it doesn’t say anything about how many steps you need per iteration. How many steps you need depends on your model and your data. Why not just try 500 iterations and time that instead?
10 leapfrog steps is very optimistic for hard models but a fine guess for very simple models. Complex models will sometimes take 10-100 steps per iteration early on, then sometimes thousands during warmup, and then fewer again during late warmup as stepsize is increased. YMMV, check your model’s n_leapfrog__ parameter. So basically this number is a useful lower bound for how long it takes to get through 1000 iterations. Don’t run 4000 iterations, you don’t need to. I often end up running 1000 warmup and 100 sampling when I’m doing development and then only up it as necessary to get final estimates once the model is working well.
The watchBigStan() function in ezStan (https://github.com/mike-lawrence/ezStan) provides a relatively naive estimate on real-time based on current progress and time taken so far.
This estimate is assuming you’re taking 10 leapfrog steps per transition for 1000 transitions.
If Stan only needs 2 leapfrog steps per iteration it’ll be a lot faster and if it needs 1000 steps per transition, it’ll be much much slower. But as far as I know, there’s no online reporting for number of transitions.
It’s very hard to estimate how many leapfrog steps will be required per iteration until after warmup has completed. At that point, the number of leapfrog steps per iteration is usually pretty stable. Before that, we’re doing blockwise adaptation and the number of steps can change dramatically.