There was a debate on one of the closed Stan Math PRs regarding the release notes. I didnt feel like having another long winded discussion so I took 15 minutes and did a first draft.

Two questions:

- Does anyone mind if we open a new wiki page where we keep an updated list of such one-liners? If you feel helping out this process just add a one-liner “Release notes” to each Math PR you open. If not, that is also fine with me, dont want to set new rules just for this. Its only 15 minutes/3 months :).
- Any objections to the draft?

Stan-user facing changes:

- added a new function to solve systems of nonlinear algebraic equations using Newton-Krylov methods.
- re-implemented a reentrant safe and scalable version of lgamma
- change the default boost_policy_t to avoid the double to long double propagation, this could affect the following functions: falling_factorial, rising_factorial, digamma and lgamma
- implemented oridinal_logistic_glm_lpmf
- fixed boundary conditions for log_sum_exp and log_diff_exp
- GPU support for the following functions: gp_exp_quad_cov, mdivide_right_tri, multiply, mdivide_left_tri, normal_id_lgm, bernoulli_logit_glm_lpmf and poisson_log_glm_lpmf

Testing:

- added a new autodiff testing framework for simpler and more automated tests
- added finite difference versions of gradient, Hessian, and gradient of Hessian functions that automatically scale step size based on input
- cleaned up test behaviour when -NDEBUG is set
- tests for bernoulli_logit, poisson_log and negbin_2_log GLMs are now tested with y as std::vector instead of Eigen::Matrix for better test coverage of the usual use case from Stan

Refactor/Code cleanup/Technical debt:

- refactor of the type traits to use more C++14 features & SFINAE
- cleaned up rev/mat/fun with the eigen plugin methods .val(), .adj() and .vi()
- typename *_return type was changed to *_return_type_t
- removed deprecated Eigen content from math headers
- metaprograms cleanups: added various enable_if_*
- added use of const ref and ref to replace copies with moves
- added clang-tidy to the makefiles
- cleaned up unnecesary Boost and Gtest compiler flags
- beta binomials lpmf loop and conditionals cleanup
- clang-tidy cleanup of the Stan Math codebase (adding braces, replace typedef with using and NULL with nullptr)
- the make test-math-dependencies file was rewritten from make to Python

OpenCL/GPU:

- GPU support for prim/gp_exp_quad_cov, prim/mdivide_right_tri, prim/multiply, rev/multiply
- fixed a bug with padding matrices
- added the triangularity view to matrix_cl
- the key for opencl kernel options is now a std::string not a const char *
- added move constructors and assigments to avoid unnecesary copying
- all references to “GPU” were changed to “CL”
- changed the copy function to to_matrix_cl and from_matrix_cl
- updated the OpenCL headers which also allowed for some OpenCL core code cleanup