I wanted to double check whether a PSIS LOO comparison is valid for a comparison between these two models. I am looking at event related potential brain activity which is often baselined by activity before the “event”. But I saw that statistical power should improve if you include the baseline term in the regression.
So let’s say the first model predicts the baselined amplitude for each participant mean:
real mu = participant_mean[participant_id_vec[i]];
amplitude[i] ~ normal(mu, sigma);
...
log_lik[i] = normal_lpdf(amplitude[i] | mu_pred[i], sigma);
Then the second model predicts the raw amplitude where the same baseline period is used as a predictor.
real mu = participant_mean[participant_id_vec[i]] + (baseline_slope*baseline_amplitude[i]);
raw_amplitude[i] ~ normal(mu, sigma);
...
log_lik[i] = normal_lpdf(raw_amplitude[i] | mu_pred[i], sigma);
I would love to compare PSIS LOO between the models to know if the baseline term does work as expected. But I am not sure a Loo comparisons would be valid because the data used to find the log_liks are different even though one is a baselined version of the other. So can Loo be used here? If not, is there some other way to still use Loo still? “sigma” is half as large in the second model, but Loo would still be helpful.
Thanks in advance! I really enjoy using the Stan software.
— Edit —
I found this forum post with a similar question, but I am struggling to understand how to apply the logic to my question. I tried this adjustment to the first model, where I find the log_liks for raw_amplitude (below). It gives the same Loo values as the original which makes sense because I am just shifting everything.
log_lik[i] = normal_lpdf(raw_amplitude[i] | (mu_pred[i] + baseline_amplitude[i]), sigma);