I am computing WAIC for an IRT model. However I cannot compute it when I run the model for 5000 iterations because the dataset is big: 13000 observations * 7 items.
It means that the running has to save or store a large matrix of size 91000 * 2500. When I run in my laptop, windows asks me to stop because of exceeding RAM memory. When I run in a HPC (High Performance Computer) I get an error:
Error in vapply(out, "[[", 2L, FUN.VALUE = numeric(1)) :
values must be length 1,
but FUN(X[]) result is length 0
Calls: loo -> loo.matrix -> psislw -> vapply
I guess the reason is the size of the big matrix because when I run with 20 iterations it is fine.
Do you have any idea to overcome the problem, how can I compute WAIC for a big dataset but with a small laptop or HPC with limited RAM?