Hi! The discussions on parallel autodiff v2 lead to a few considerations we really would like to run the parallel phase to also run during the reverse sweep we concentrate on the reduce sum only => so only a single output we want to keep track of all var’s which are part of anything running paral…

I think @t4c1 's apply implementation should be helpful stan-dev/math/blob/develop/stan/math/prim/meta/index_apply.hpp #ifndef STAN_MATH_PRIM_META_INDEX_APPLY_HPP #define STAN_MATH_PRIM_META_INDEX_APPLY_HPP #include <utility> #include <cstddef> namespace stan { namespace math { namespace …

[image] stevebronder: For the tuples stuff theres apply in the std with an implementation here That’s C++17, isn’t it?

Yes but they have an implementation example we could use

Yo do you mind if I push stuff into your branch?

‘You’ here being @stevebronder

Alright, stuff is up ( https://github.com/stan-dev/math/compare/proto-parallel-v3...cleanup/proto-parallel-v3 ). I screwed up autodiff on the sliced arguments. I was scared about what would happen if all the vars in the sliced arguments had the same vari, so I just didn’t implement that. I broke…

This should allow for Stan arguments real, int, real[], int[], vector, row_vector, matrix to be passed as the shared arguments. This stuff is probably pretty broken. I didn’t test the new stuff and there’s plenty of places for this to break lol.

Nice! I’ll try to take a look at this tonight. I made a WIP PR because I think it’s easier to talk about code over on github [WIP] Parallel Prototype stan-dev:develop ← stan-dev:cleanup/proto-parallel-v3 opened 01:03AM - …

I got enable ifs up and running so we only have one reduce_sum_impl

Parallel autodiff v3

Developers

stevebronder January 14, 2020, 11:54pm 40

Yeah go it!

Topic		Replies	Views
Parallel autodiff v4 Developers	97	4873	June 20, 2023
Proposed parallelism RFC - Stan language bits Developers	14	1196	July 9, 2019
Parallel autodiff v2 Developers math	29	2169	December 31, 2019
Parallel reduce in the Stan language Developers	12	1366	April 11, 2019
(older) parallel AD tape ideas Developers	42	1589	November 4, 2020

Parallel autodiff v3

Related topics