The model’s a bit too much to dive through, but here’s a few things to keep in mind:
-
Stan is going to store all the parameters and transformed parameters every iteration—this can get big. You can work out how big this is by counting out 8 bytes or so for each value saved. There are a lot of 4 x 5 x N_s_predictors and
4 x 5 x N_d_predictors. -
Every intermediate computation result consumes about 40 bytes on the autodiff stack. So this is likely to be where you run out of memory. This gets reused each iteration, but it can also consume a lot of memory.
-
You need to make sure Stan gets a lot of memory access. Do you have a 64-bit OS and how much memory are you giving it now?
I’m guessing from looking at your code that the main problem is (2). If you use something like CmdStanPy or CmdStan, it streams output to disk. So that might be a way to get it to run without blowing out memory. Then you can fire up a new job to read the data back in.
Bonus. I’d try very hard to not use exp if you can avoid it. It’s very numerically unstable. The longe you can stay on the log scale, the more stable things will be. Instead of adding, use log_sum_exp and instead of multiplying, add, and instead of exponentiation, multiply.