Line search failed error

Algorithms
1

I’m trying to use the optimizer (lbfgs and bfgs) and keep getting the following error: (these are the final lines of the cmdstan output)

   Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
     189       1248.86     0.0257983       27.3849           1           1      242   
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
     190       1248.87   1.87757e-05       23.6615   6.856e-07       0.001      286  LS failed, Hessian reset 
    Iter      log prob        ||dx||      ||grad||       alpha      alpha0  # evals  Notes 
     191       1248.87   1.87757e-05       23.6615   5.206e-11       0.001      357  LS failed, Hessian reset 
Optimization terminated with error: 
  Line search failed to achieve a sufficient decrease, no more progress can be made

Does anyone have any advice on how to avoid this? The model is quite complex and uses and uses the ODE solver. Even though the optimizer terminates with an error, the fit goes through the data quite well. Is there perhaps a way to tweak the tolerances in some way? I’m using the default values.

2

Hi @chvandorp, welcome. You say you are using CmdStan. From within the git cloned cmdstan directory, you can run make manual to build the CmdStan manual – I believe there is a dependency on latexmk. Within Section 9.5 of the manual, check out “Optimization-Specific Commands” to learn about the arguments you can tweak. Good luck.

3

The line search error usually happens when the computed log prob or gradient have some error and the search direction does not point downhill. ODE solver is iterative and has its own tolerance affecting the accuracy. In your case it seems that the optimization is very near the optimum as log prob and ||grad|| are not changing much, and ||dx|| is small. Very small alpha (related to step size in line search) in Iter 191 supports the assumption that the search direction is not pointing downhill.

See “Optimization-Specific Commands” in CmdStan user guide

You can also test running with smaller ODE solver tolerances whether there is any practical difference in the optimum found.

1 Like
4

Hi @avehtari
Solving the ODEs with smaller tolerances works (9/10 times, which is better than 0/10 times).
Thanks!