Ideas for getting better estimates of expected sampling duration?

General
1

One of the features of ezStan is an attempt to provide improved indicators of sampling progress, including an estimate of remaining time until completion. This is achieved simply by keeping track of when sampling started and watching the csv files of samples to compute the samples/wall-time and thereby time remaining.

However, I find that the warmup period often takes much longer to achieve each sample than the post-warmup period, throwing off my estimates. Is there anything in the sample files that might help me estimate the degree to which the post-warmup will be faster than warmup and thereby adjust the remaining-time estimate accordingly? For example, maybe something to do with stepsize__, treedepth__ or n_leapfrog__, all of which seem much more volatile during warmup than post-warmup? Any ideas?

2

No – the adaptation in warmup is a highly nonlinear process. There’s a reason why we don’t have estimated time output in Stan right now.

The initial projection warning is simply a way to quantify the cost of the gradient and provide an estimate based on the asymptotic performance for reasonably nice geometries. For complex geometries it is a significant underestimate, but there’s now way to quantify that until the model has already been run.