Trying to fit my costumised logistic function

[martinmodrak]

@erognli

What I think you should be doing (I trust someone will correct me if I’m giving you bad advice now) is to use log_sum_exp to sum the log-likelihoods of each participants trials, and then use that in loo.

Thanks for being helpful, the overall idea is sensible, but the execution is unfortunately incorrect - to have leave-one-participant-out cross validation, you sum the individual log-likelihoods directly. This actually could lead to a more compact code, as you can then have (ignoring generating predictions):

generated quantities {
  real log_lik[N];

    for(n in 1:N) {      
      log_lik[n] = binomial_lpmf(y[n,] | trials[n, ], performance[n, ]);
    }
}

Some more discussion at:

Leave-one-group-out CV

I hope you don’t mind me hijacking the thread for a question about leave-one-group-out cross-validation that’s been bugging me. Suppose that I want to model N observations nested in J groups (for example, N responses from J participants in a psychological study). Going back to @avehtari 's blog post (https://andrewgelman.com/2018/08/03/loo-cross-validation-approaches-valid/), this example would correspond to the school rather than state case, and one would want to estimate the model’s predictio…

@Carina

I guess the looic behaves as the aic, and thus is in itself not informative? My ppc looks pretty weird

I think (not 100% sure) that elpd_loo can be a somewhat meaningful quantity (it is the “expected log predictive density”), but generally I find interpretation without another model to compare to challenging. PPC’s are in my experience much more useful to asses fit of a single model.

Hope that helps!