Data reformatting and storing / function memoization

Developers
1

Hi!

I was wondering if we have facilities (some singleton or similar) which allows immutable storage of the user input data in shapes more suitable for individual functions.

The idea is simple: The user passes in the data into Stan programs as he has to given our functions. Now, the functions then have to take the input data and possibly reformat it or maybe derive quantities from this. Currently, these operations which act on user data will have to be redone every-time the function is called despite the fact that the user data never changes. These manipulations are certainly cheap compared to all the rest which is going on, but avoiding this should be possible by, for example, letting functions register and save the transformed data operations somewhere.

This concept is also known as memoization as far as I am aware. Do we have this? Should we have this? Is it easy?

Sebastian

2

If it’s really data and can be done once, put it in the transformed data block.

“Memoization” is the word for storing intermediate computations for reuse. If you look at the HMM dynamic programming algorithm (the “forward” algorithm), that uses a kind of memoization to turn what would be a naive exponential algorithm into a linear one.

I don’t think we want to have stateful functions.

What I do think we want longer term is closures, so that we don’t have to bother reshaping variables but can just access them as defined in the data block or parameters block from within functions. That’s also hard to reason about, but I don’t think it’s quite as nasty as having stateful functions.

1 Like