Hi! I just speeded up a model by a factor of 10x by replacing a matrix-vector product with a loop which only processes the non-zero elements of the matrix. The matrix was a design matrix (thus the matrix was data for Stan) for the random effects part of a model. I assume that this is the same situa…

There’s a fix in R-Stan ( @bgoodri did it) and I can write one into math if it’s needed (the derivatives are not hard). How much of a pain would it be to swap your code to column-major? Because that’s more useful for other sparse operations…

Thanks @sakrejda for the link. I recall @bgoodri did come up with some proposals…is that documented somewhere? Doing something about these sparse matrices would be a huge win. Like I say, I got a problem to sample 10x as fast and the matrices were not really large, but with massive amounts of 0s. …

stan-dev/rstanarm/blob/master/inst/include/csr_matrix_times_vector2.hpp /* * This works exactly like csr_matrix_times_vector but faster and less safe */ template <typename T1, typename T2> inline Eigen::Matrix<typename boost::math::tools::promote_args<T1, T2>::type, Eigen::Dy…

The original reason the code was made as slow as it is was opposition to giving users a foot-gun so adding Ben’s version to math would involve re-hashing that. Once tuples are in I think we could build a type on top of them that would carry the sparse matrix component vectors and be constructed in …

Hmmm… i recall. Have we considered adding an explicit unsafe version? If the unsafe is made clear in the function name, then one could possibly skip error checking, no? Has that idea been discussed? I don‘t think it is ideal to have critical speed fixes in rstanarm alone and tuples are still some b…

It wasn’t a huge change in speed. It might have actually been slower in doubles, but we gained a bit more speed by relying on the autodiff instead of precomputed_gradients.

[image] wds15: So it seems to be a major problem as the AD stack gets inflated by all those zero times whatever product. Yes. That’s why I always tell people not to do that and why we have a sparse matrix-vector multiply function. Getting sparse matrices is backed up behind tuples, function…

When you say proposal what exactly do you mean? Is there an example for, say, ragged arrays (or MPI or ODEs or something else recent)?

[image] anon75146577: proposal what exactly do you mean? Is there an example for, say, ragged arrays (or MPI or ODEs or something else recent)? Ragged arrays: Home · stan-dev/stan Wiki · GitHub It’s not particularly coherent because I just pasted a new proposal in front of the old one (ever…

Sparse matrices in Stan

Developers

Krzysztof_Sakrejda January 29, 2018, 6:32pm 2

Latest, with links back into history: A proposal for sparse matrices and GPs in Stan

Topic		Replies	Views
Sparse model matrix functionality without the sparse matrix Modeling techniques	3	1015	September 25, 2018
A proposal for sparse matrices and GPs in Stan Developers	35	5340	January 18, 2018
Should sparse matrices in Stan be row major or column major Developers	47	5193	August 21, 2017
Sparse matrix roadmap Developers	37	2081	October 8, 2018
Sparse matrix use cases (resolving declaration issues) Developers	22	3775	August 16, 2017

Sparse matrices in Stan

Related topics