Increasing BFGS (optimization) iterations in MATLAB-Stan

Modeling
1

Dear Authors and Developers,

We have been using Stan (MATLAB interface) for training our models. We find it very easy to write and optimize the log posterior.

However, We find the ‘iter’ parameter unconfigurable for the optimizers (BFGS and L-BFGS) and is always set to a default 2000. Is there something we can do to increase over this default value, as we would really like more iterations for convergence in our recent applications.

I am trying to change the number of iterations here to 5000:

op = stan(‘file’, ‘FLDCRF_opportunity3.stan’, ‘method’, ‘optimize’, ‘data’, dat, ‘algorithm’, ‘bfgs’, ‘iter’, 5000, ‘verbose’,true);

But, it looks like this parameter ‘iter’ is only valid for sampling, and the BFGS optimization stops after 2000 iterations by default, whether or not an optima. I tried editing the ‘stan.m’ file as well as below:

"
p = inputParser;
p.KeepUnmatched= true;
p.FunctionName = ‘stan’;
p.addParamValue(‘fit’,[],@(x) isa(x,‘StanFit’) || isa(x,‘StanModel’));
p.addParamValue(‘method’,‘optimize’);
p.addParamValue(‘iter’, 5000,@(x) isscalar(x) && (x>0));
p.addParamValue(‘warmup’,[],@(x) isscalar(x) && (x>0));
p.addParamValue(‘refresh’,[],@(x) isscalar(x) && (x>0));
p.addParamValue(‘algorithm’,’’);
"

with the same result. Is it possible to run more than 2000 iterations?

Looking forward to your response.

Thanks and Regards,
Satyajit

2

All of that is configurable in Stan and the other interfaces, so I think it is an issue with MatlabStan or how you are calling it.

3

Thanks for your response. Is it possible to communicate the proper way of setting it through MatlabStan? (preferable)

Alternately, What would be the way to set it through PyStan or the command line?

Thanks,
Satyajit

4

Actually, it seems that everything is configurable except the iteration limit. You would have to change it in the source code

5

Thanks very much.

I have updated the ‘defaults.hpp’ under optimize (to 5000) and recompiled the stan code. The program is running now.

"
struct iter {
/**
* Return the string description of iter.
*
* @return description
*/
static std::string description() {
return “Total number of iterations.”;
}

    /**
     * Validates iter; iter must be greater than 0.
     *
     * @param[in] iter argument to validate
     * @throw std::invalid_argument unless iter is greater than zero
     */
    static void validate(int iter) {
      if (!(iter > 0))
        throw std::invalid_argument("iter must be greater than 0.");
    }

    /**
     * Return the default iter value.
     *
     * @return 2000
     */
    static int default_value() {
      return 5000;
    }
  };

"
However, the display iter = 2000 (default) doesn’t change, may be this is getting printed from somewhere else?
"
output.csv: tol_rel_grad = 10000000 (Default)
output.csv: tol_param = 1e-008 (Default)
output.csv: iter = 2000 (Default)
output.csv: save_iterations = 0 (Default)
output.csv: id = 1
output.csv: data
output.csv: file = C:\Satyajit\StanModellingLlanguage\MatlabStan-2.7.0.0\MatlabStan-2.7.0.0\temp.data.R
output.csv: init = 2 (Default)
output.csv: random
output.csv: seed = 210712
output.csv: output
output.csv: file = output.csv (Default)
output.csv: diagnostic_file = (Default)
output.csv: refresh = 250
"

I will update once the run finishes. Please let me know if I should make any other obvious changes.

Regards,
Satyajit

6

Hello Sir,

The run has finished and again terminated after 2000 iterations.

"
output.csv: iter = 2000 (Default)
output.csv: save_iterations = 0 (Default)
output.csv: id = 1
output.csv: data
output.csv: file = C:\Satyajit\StanModellingLlanguage\MatlabStan-2.7.0.0\MatlabStan-2.7.0.0\temp.data.R
output.csv: init = 2 (Default)
output.csv: random
output.csv: seed = 215035
output.csv: output
output.csv: file = output.csv (Default)
output.csv: diagnostic_file = (Default)
output.csv: refresh = 250
output.csv: Initial log joint probability = -2.42469e+008
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 249 -81414.9 0.0252592 173967 0.1968 1 292
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 499 -43350.9 0.000751868 73704.1 0.3109 0.3109 598
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 749 -34687.4 0.0268434 66399.1 1 1 895
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 999 -27739.5 0.00584042 133084 0.9123 0.9123 1201
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 1249 -25051.6 0.000951491 54264 1 1 1503
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 1499 -24130.1 0.00291518 25619.2 1 1 1803
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 1749 -22862.3 0.00711312 62576.5 1 1 2092
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 1999 -22289.8 0.00494965 29760.1 0.2993 1 2405
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 2000 -22289.1 0.00125429 27343.9 0.2968 0.2968 2406

output.csv: video1
output.csv: time 1::Layer 1 locomotion:stand: 0:walk: 0:sit: 1:lie: 0

"

Could there be other places in the source code I need to update?

Regards,
Satyajit

7

@satyajit001, I’ve got a working hypothesis about what’s going on here, but no matlab to try it out on. Would you try editing line 116 of the file MatlabStan/+mstan/stan_params.m on your machine and then re-running? Let me know what you get. Thanks.

8

Thanks very much @roualdes . It has crossed 2000 iterations and running!

"
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes
output.csv: 249 -72313.3 0.0366945 121219 0.9477 0.9477 276
output.csv: 499 -31613.4 0.0680172 35445.4 1 1 565
output.csv: 749 -20576.5 0.00634662 99858.8 0.7919 0.7919 847
output.csv: 999 -16212.1 0.0074516 33723.4 0.2962 0.2962 1117
output.csv: 1249 -13404.7 0.00628656 21676.8 0.7098 0.7098 1384
output.csv: 1499 -11641.4 0.0143443 23108.4 1 1 1648
output.csv: 1749 -10443.3 0.00513589 30005 0.6458 0.6458 1926
output.csv: 1999 -9781.29 0.00261364 5619.61 1 1 2197
output.csv: 2249 -9524.05 0.0265424 15546.9 0.6901 0.6901 2473
"

Regards,
Satyajit

9

Thanks for checking that out and reporting back.

Just for the record, this appears to be an issue in MatlabStan. The documentation really means, the iter option is only valid for the method sample:

% iter - scalar, optional, valid when method = 'sample'

The value of iter used for optimizing comes from the file MatlabStan/+mstan/stan_params.m not from stan.m.