Hi all,
Fairly new to RStan, but having some success with it running MaxDiff studies (using the flipMaxDiff package in R).
But as a Sawtooth user I’m used to being able to easily obtain a respondent level Root-likelihood (RLH) score which I can then use for data cleaning for a second iteration of the analysis.
Does anyone know how to obtain that from a stanfit object, if it exists?
Failing that, are there different (respondent level) fit statistics that I can use in a similar way?
As a follow-up to this… I’ve just come across the prediction.accuracies in my stan fit object.
I’m wondering if I can use these for the same purpose I would have used RLH in the past, but can’t find any direct reference to these in the documentation e.g., on Runtime warnings and convergence problems
Can someone link me to some reading on what prediction accuracies represent?
I don’t know what an RLH is. Stan doesn’t compute it automatically, but you can probably do it manually if you have a definition. You might need to put it into the Stan model if it’s not something that’s computable from posterior draws.
I’m not sure what that is. Are you using brms or some other package built on top of Stan?
Hi Bob,
Yes, I’m using the R package flipMaxDiff on top of stan.
Once the model has run, this results in an object of class ‘FitMaxDiff’ which includes a list of various outputs. The prediction.accuracies are output as part of that.
The FitMaxDiff object also includes an object called stan.fit (of class ‘stanfit’) which includes many different elements (including the standard model fit statistics rhat, ess, etc) so if there are respondent level fit statistics available in the standard stan output I can probably access them - I just don’t know what they are called as stan is so new to me.
The flipMaxDiff package seems ot have limited documentation so I was hoping someone here might be more familiar with it, but no worries if not.