@paul.buerkner I’m working on this to help, but I have to get up to speed on brms itself first. I’m trying to generate the stan code, but it seems brms::make_stancode doesn’t simply spit out code, but also runs the syntax checker and fails to show you any code if there’s a syntax error (as there is in Jon’s model code). Any way I can generate code without bothering to check syntax?
make_stancode(…, testmode = TRUE)
1 Like
The error must be due to an update – it ran at the time of my posting. I think the solution is probably to add a vector for threshold into the function definition as suggested by @paul.buerkner earlier.
OK, I think I’ve wrapped my head around this now and am following Paul’s suggestion to put the mixture model inside a custom family. @paul.buerkner: the brms::custom_family arguments includes a threshold variable for ordinal families, from which I presume brms will take care of setting up the thresholds when it sees that an ordinal family is being used. Is there an example of a custom ordinal family specification anywhere? As I’m guessing when creating these there must be some standard naming of the thresholds for use inside the function.
@paul.buerkner, I found some time to implement your recommendation and developed a custom family rather than using the mixture interface. I now possess a variant of my model capable of recovering parameters from simulated data and am trying to implement the supporting functions (i.e., log_lik_, predict_, fitted_, _rng). I have two follow-up questions:
-
With respect to the supporting functions, I am struggling. I tried following the tutorial, but my application appears to be more complex. For example, I had to figure out via trial and error that thres (which is a parameter) was under neither dpars nor data within the draws structure (but rather accessed via
draws$thres). It would be an enormous help if I could instead re-create the precisedrawsobject being handed to these functions so I could work with them directly. Is there an easy way to generatedraws? -
When trying to compare my new model specification to other models using bayes_factor, I receive the following error. Any advice as to how I might approach fixing it? I am unsure whether this is a
brmsorbridgesamplingquestion.
Error in tmp$r_vals[lr - 1] * tmp$r_vals[lr] :
non-numeric argument to binary operator
In addition: Warning messages:
1: 7 of the 20000 log_prob() evaluations on the proposal draws produced -Inf/Inf.
2: logml could not be estimated within maxiter, rerunning with adjusted starting value.
Estimate might be more variable than usual.
Error: Bridgesampling failed. Did you set ‘save_all_pars’ to TRUE when fitting your model?
-
Use extract_draws and look in the brms code base on github for examples of how ordinal families work with these draws.
-
I have no idea. Could be happening inside rstan or inside bridgesampling.
That’s an old question, but maybe it’d be helpful for others. I also had the error described in (2). It turned out, that the priors I was using were very far from the real values. Setting more adequate priors helped.
1 Like
Funny, I’ve been trying for the past two days to implement a two-process recollection+familiarity model of confidence judgements in brms and after running into an error and finding your post, I immediately recognize that you were trying to do the same. @Jon_Fawcett, did you ever get it to work and would you mind sharing the code? No point in reinventing the wheel.
1 Like
I ended up turning to the author of this package who then implemented the model GitHub - boryspaulewicz/bhsdtr2: Bayesian hierarchical ordinal models with ordered thresholds
Hope that helps! I spent more time trying to work out how to fit that model than I care to admit! Borys was very responsive, so I suspect if you ran into trouble and reached out, he would reply. Good luck!
2 Likes