Default initial step size

Algorithms
1

What is the default initial step size used by:

  • RStan (element stepsize in the control argument of rstan::stan())?
  • CmdStan (and therefore also by CmdStanR, argument step_size of the $sample() method for CmdStanModel objects)?

I can’t find these default values in the corresponding docs. The only document I’ve found is this one where a value of 2.0 is mentioned, but that document only seems to refer to RStan. In the CmdStan source code, a value of 1 seems to be used, but I’m not sure if I’m interpreting the source code correctly.

2

Hi,

I assume they use the default values Stan uses?

i.e., \delta=0.8, \gamma=0.05, \kappa=0.75, and t_0 = 10. Is this what you were after?

3

There’s is no default stepsize per se, but only default method to initialize stepsize, which you can find in NUTS paper. The gist is to come up a stepsize that makes the acceptance probability above given threshold. The actual value of initial stepsize varies by random seed. The code you are looking for is init_stepsize in base_hmc.hpp.

EDIT: I changed the category to “algorithm”.

4

Thanks a lot for your replies.

First, I’m sorry because I didn’t realize that there is a special CmdStan Guide. There, on page 56, it explicitly says that the default value for stepsize is 1. Then I guess this default is also used for RStan?

Secondly, I think I understood @yizhang’s explanation. However, what is then the purpose of this argument stepsize (or step_size in CmdStanR)? My whole question arose in the first place because I was getting several informational messages by CmdStanR saying

Scale parameter is inf, but must be finite!

and I realized that I can get rid of these by lowering the step_size value to 0.1, as proposed here.

5

What’s given to step_size arg is exactly for init_stepsize, a starting point to find a proper stepsize for adaptation to get to proper acceptance probability. That step_size defaults to 1(cmdstan user guide), and reducing it makes it easier to reach that acceptance probability(so you don’t see those warnings anymore). Just keep in mind that step_size is not the actual stepsize the adaptation uses in the first warmup iteration.

3 Likes
6

Alright, thanks.

7

This is correct. Note also that if step_size is small enough then the step size initialization routine will immediately pass and sampling will start with a smaller step size than what the initialization routine would have found even if it had worked, so it can be a helpful way to facilitate warmup in difficult problems.

4 Likes
8

I forgot to ask again for the default value used by RStan: Is this also 1 (in contrast to this document already mentioned above)?

9

Yes it’s 1 by default, as it’s basically cmdstan specification. See https://mc-stan.org/docs/2_23/cmdstan-guide/mcmc-config.html#hmc-samplers

1 Like
10

This is missing from the RStan doc, so I created an issue to add it there:

1 Like