Thanks Bob. Your approach is much better, and thanks for the reminder to compute the log of posterior mean. Two questions for anyone…

(1) To use loo approximations, I would need to get log_lik terms at the group level. Anyone know a good way to sum the log_lik terms by group without storing the full pointwise log_lik?

(2) The loo approximations may not work for my group level prediction task (i.e. small number of groups (20) with high inter-group variability), so I may still need leave-one(or many)-group-out CV. I haven’t quite got the above @Bob_Carpenter approach to work when I hold out entire groups in a multilevel model because of the way I index the group parameters (e.g. jj[i])…has anyone done something like @Bob_Carpenter is suggesting while leaving groups out? I’ll report back if I figure it out.

For now I’ve been creating separate training and test data outside of Stan and redefining the Stan model with fewer group parameters (see linear regression example below with repeated measurements by person and one person held out). Then I calculate the log_lik without the person level parameters. Is this correct leave-one-group-out CV?

```
data{
// training set
int<lower=1> N; // num records
int<lower=1> J; // num participants (groups)
int<lower=1> K; // num predictors
int<lower=1,upper=J> jj[N]; // participant index
real y[N]; // outcome
// testing set
int<lower=1> N_test;
real y_test[N_test];
}
parameters{
vector[J] a_person; // varying ints by person
real a;
real<lower=0> sigma_person;
real<lower=0> sigma;
}
model{
sigma ~ student_t( 3 , 0 , 2 );
sigma_person ~ student_t( 3 , 0 , 2 );
a ~ normal( 0 , 4 );
a_person ~ normal( 0 , sigma_person );
for ( i in 1:N ) {
y[i] ~ normal( a + a_person[jj[i]], sigma);
}
}
generated quantities{
// log_lik for held out person
real log_lik = normal_lpdf(y_test | a , sigma ); // no a_person term
}
```