In the “things we should discuss” category for sparse matrices, we need to think about whether they should be stored in row or column major formats. Here’s what currently exists:
– Stan math uses this in its one supported sparse matrix operation
– My memory is that reverse autodiff for the cholesky used to use row-major ordering because @rtrangucci implemented Giles’ algorithm which happened to be coded that way (or that’s my view from the old doc). This has been superseded, but it may be a consideration when we put the sparse functionality in properly (including a var specialisation of sparse cholesky). That being said, there’s no real reason we can’t re-interpret Smith’s algorithm in row-major form (he didn’t write it in either direction) and there’s a moderately good chance it will be tricky to program regardless of the choice between row-major and column-major.
– This is the default for Eigen.
So is there any compelling reason to choose one over the other, or has this just been a series of historical quirks?