# Increasing BFGS (optimization) iterations in MATLAB-Stan

**URL:** <https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150>\
**Category:** Modeling\
**Tags:** matlabstan\
**Created:** [March 19, 2019, 9:50pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150 "2019-03-19T21:50:21Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![satyajit001](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@satyajit001](https://discourse.mc-stan.org/u/satyajit001)\
**Post date:** [March 19, 2019, 9:50pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/1 "2019-03-19T21:50:21Z")

</div>

Dear Authors and Developers,

We have been using Stan (MATLAB interface) for training our models. We find it very easy to write and optimize the log posterior.

However, We find the ‘iter’ parameter unconfigurable for the optimizers (BFGS and L-BFGS) and is always set to a default 2000. Is there something we can do to increase over this default value, as we would really like more iterations for convergence in our recent applications.

I am trying to change the number of iterations here to 5000:

op = stan(‘file’, ‘FLDCRF\_opportunity3.stan’, ‘method’, ‘optimize’, ‘data’, dat, ‘algorithm’, ‘bfgs’, ‘iter’, 5000, ‘verbose’,true);

But, it looks like this parameter ‘iter’ is only valid for sampling, and the BFGS optimization stops after 2000 iterations by default, whether or not an optima. I tried editing the ‘stan.m’ file as well as below:

"  
p = inputParser;  
p.KeepUnmatched= true;  
p.FunctionName = ‘stan’;  
p.addParamValue(‘fit’,[],@(x) isa(x,‘StanFit’) || isa(x,‘StanModel’));  
**p.addParamValue(‘method’,‘optimize’);**  
**p.addParamValue(‘iter’, 5000,@(x) isscalar(x) && (x\>0));**  
p.addParamValue(‘warmup’,[],@(x) isscalar(x) && (x\>0));  
p.addParamValue(‘refresh’,[],@(x) isscalar(x) && (x\>0));  
p.addParamValue(‘algorithm’,’’);  
"

with the same result. Is it possible to run more than 2000 iterations?

Looking forward to your response.

Thanks and Regards,  
Satyajit

---

<div class="post-metadata">

**Author:** ![bgoodri](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bgoodri/32/4451_2.png) [@bgoodri](https://discourse.mc-stan.org/u/bgoodri)\
**Post date:** [March 19, 2019, 9:57pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/2 "2019-03-19T21:57:43Z")

</div>

All of that is configurable in Stan and the other interfaces, so I think it is an issue with MatlabStan or how you are calling it.

---

<div class="post-metadata">

**Author:** ![satyajit001](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@satyajit001](https://discourse.mc-stan.org/u/satyajit001)\
**Post date:** [March 19, 2019, 10:05pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/3 "2019-03-19T22:05:41Z")

</div>

Thanks for your response. Is it possible to communicate the proper way of setting it through MatlabStan? (preferable)

Alternately, What would be the way to set it through PyStan or the command line?

Thanks,  
Satyajit

---

<div class="post-metadata">

**Author:** ![bgoodri](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/bgoodri/32/4451_2.png) [@bgoodri](https://discourse.mc-stan.org/u/bgoodri)\
**Post date:** [March 19, 2019, 10:31pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/4 "2019-03-19T22:31:45Z")

</div>

Actually, it seems that everything is configurable except the iteration limit. You would have to change it in the source code

> <https://github.com/stan-dev/stan/blob/develop/src/stan/services/optimize/defaults.hpp#L294>

---

<div class="post-metadata">

**Author:** ![satyajit001](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@satyajit001](https://discourse.mc-stan.org/u/satyajit001)\
**Post date:** [March 19, 2019, 11:50pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/5 "2019-03-19T23:50:09Z")

</div>

Thanks very much.

I have updated the ‘defaults.hpp’ under optimize (to 5000) and recompiled the stan code. The program is running now.

"  
struct iter {  
/\*\*  
\* Return the string description of iter.  
\*  
\* @return description  
\*/  
static std::string description() {  
return “Total number of iterations.”;  
}

```
    /**
     * Validates iter; iter must be greater than 0.
     *
     * @param[in] iter argument to validate
     * @throw std::invalid_argument unless iter is greater than zero
     */
    static void validate(int iter) {
      if (!(iter > 0))
        throw std::invalid_argument("iter must be greater than 0.");
    }

    /**
     * Return the default iter value.
     *
     * @return 2000
     */
    static int default_value() {
      return 5000;
    }
  };

```

"  
However, the display iter = 2000 (default) doesn’t change, may be this is getting printed from somewhere else?  
"  
output.csv: tol\_rel\_grad = 10000000 (Default)  
output.csv: tol\_param = 1e-008 (Default)  
output.csv: iter = 2000 (Default)  
output.csv: save\_iterations = 0 (Default)  
output.csv: id = 1  
output.csv: data  
output.csv: file = C:\Satyajit\StanModellingLlanguage\MatlabStan-2.7.0.0\MatlabStan-2.7.0.0\temp.data.R  
output.csv: init = 2 (Default)  
output.csv: random  
output.csv: seed = 210712  
output.csv: output  
output.csv: file = output.csv (Default)  
output.csv: diagnostic\_file = (Default)  
output.csv: refresh = 250  
"

I will update once the run finishes. Please let me know if I should make any other obvious changes.

Regards,  
Satyajit

---

<div class="post-metadata">

**Author:** ![satyajit001](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@satyajit001](https://discourse.mc-stan.org/u/satyajit001)\
**Post date:** [March 20, 2019, 1:10am UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/6 "2019-03-20T01:10:18Z")

</div>

Hello Sir,

The run has finished and again terminated after 2000 iterations.

"  
output.csv: iter = 2000 (Default)   
output.csv: save\_iterations = 0 (Default)   
output.csv: id = 1   
output.csv: data   
output.csv: file = C:\Satyajit\StanModellingLlanguage\MatlabStan-2.7.0.0\MatlabStan-2.7.0.0\temp.data.R   
output.csv: init = 2 (Default)   
output.csv: random   
output.csv: seed = 215035   
output.csv: output   
output.csv: file = output.csv (Default)   
output.csv: diagnostic\_file = (Default)   
output.csv: refresh = 250   
output.csv: Initial log joint probability = -2.42469e+008   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 249 -81414.9 0.0252592 173967 0.1968 1 292   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 499 -43350.9 0.000751868 73704.1 0.3109 0.3109 598   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 749 -34687.4 0.0268434 66399.1 1 1 895   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 999 -27739.5 0.00584042 133084 0.9123 0.9123 1201   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 1249 -25051.6 0.000951491 54264 1 1 1503   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 1499 -24130.1 0.00291518 25619.2 1 1 1803   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 1749 -22862.3 0.00711312 62576.5 1 1 2092   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 1999 -22289.8 0.00494965 29760.1 0.2993 1 2405   
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes   
output.csv: 2000 -22289.1 0.00125429 27343.9 0.2968 0.2968 2406

| output.csv: video1 | | | | | |
| --- | --- | --- | --- | --- | --- |
| output.csv: time 1::Layer 1 locomotion:stand: 0:walk: 0:sit: 1:lie: 0 | | | | | |

"

Could there be other places in the source code I need to update?

Regards,  
Satyajit

---

<div class="post-metadata">

**Author:** ![roualdes](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/roualdes/32/1594_2.png) [@roualdes](https://discourse.mc-stan.org/u/roualdes)\
**Post date:** [March 20, 2019, 5:53am UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/7 "2019-03-20T05:53:58Z")

</div>

@satyajit001, I’ve got a working hypothesis about what’s going on here, but no matlab to try it out on. Would you try editing line 116 of the file `MatlabStan/+mstan/stan_params.m` on your machine and then re-running? Let me know what you get. Thanks.

---

<div class="post-metadata">

**Author:** ![satyajit001](https://avatars.discourse-cdn.com/v4/letter/s/c4cdca/32.png) [@satyajit001](https://discourse.mc-stan.org/u/satyajit001)\
**Post date:** [March 20, 2019, 12:36pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/8 "2019-03-20T12:36:33Z")

</div>

Thanks very much @roualdes . It has crossed 2000 iterations and running!

"  
output.csv: Iter log prob ||dx|| ||grad|| alpha alpha0 # evals Notes  
output.csv: 249 -72313.3 0.0366945 121219 0.9477 0.9477 276  
output.csv: 499 -31613.4 0.0680172 35445.4 1 1 565  
output.csv: 749 -20576.5 0.00634662 99858.8 0.7919 0.7919 847  
output.csv: 999 -16212.1 0.0074516 33723.4 0.2962 0.2962 1117  
output.csv: 1249 -13404.7 0.00628656 21676.8 0.7098 0.7098 1384  
output.csv: 1499 -11641.4 0.0143443 23108.4 1 1 1648  
output.csv: 1749 -10443.3 0.00513589 30005 0.6458 0.6458 1926  
output.csv: 1999 -9781.29 0.00261364 5619.61 1 1 2197  
output.csv: 2249 -9524.05 0.0265424 15546.9 0.6901 0.6901 2473  
"

Regards,  
Satyajit

---

<div class="post-metadata">

**Author:** ![roualdes](https://yyz2.discourse-cdn.com/flex030/user_avatar/discourse.mc-stan.org/roualdes/32/1594_2.png) [@roualdes](https://discourse.mc-stan.org/u/roualdes)\
**Post date:** [March 20, 2019, 3:57pm UTC](https://discourse.mc-stan.org/t/increasing-bfgs-optimization-iterations-in-matlab-stan/8150/9 "2019-03-20T15:57:14Z")

</div>

Thanks for checking that out and reporting back.

Just for the record, this appears to be an issue in MatlabStan. The documentation really means, the `iter` option is only valid for the method `sample`:

`% iter - scalar, optional, valid when method = 'sample'`

The value of `iter` used for `optimizing` comes from the file `MatlabStan/+mstan/stan_params.m` not from `stan.m`.